High Scale QoS IOM in ESM Context: Single SLA Mode
This chapter describes the High Scale QoS (HSQ) IOM in the Enhanced Subscriber Management (ESM) context.
Topics in this chapter include:
Applicability
This chapter describes the High Scale QoS (HSQ) IOM in the Enhanced Subscriber Management (ESM) context. The information and configuration in this chapter are based on SR OS Release 15.0.R4.
Overview
This chapter describes the QoS operation and configuration of the High Scale QoS (HSQ) IOM, with a focus on single SLA mode in the Enhanced Subscriber Management ESM context.
Single SLA mode on HSQ is characterized by a single SLA profile instance (SPI) per subscriber. The subscriber can have multiple hosts or sessions, which are all sharing this single SPI. Single SLA mode can be enabled per subscriber and, while enabled, any attempt to establish more than one SPI per subscriber will be rejected by the system.
The default SLA mode on HSQ IOM is expanded SLA mode, which supports multiple SPIs per subscriber. This topic is covered in chapter High Scale QoS IOM in ESM Context: Expanded SLA Mode.
HSQ IOM in the context of ESM provides the following benefits:
Traffic management functionality on egress with seven tiers of shaping hierarchy, six strict priority scheduling levels, and weighted round-robin (WRR) groups at the subscriber and port level.
Increased scale with 786k egress queues.
Aggregate throughput in the range of 200 Gb/s full-duplex per HSQ IOM.
~96k subscribers per HSQ IOM in 1:1 (sub/SAP) scenario, each with 8 egress queues in single SLA mode.
With HSQ IOM, the ESM continues to support existing SAP deployment models in the following way:
Subscriber per SAP (1:1) – single SLA mode and expanded SLA mode
Multiple subscribers per SAP (N:1) – single SLA mode and expanded SLA mode
Multiple SAPs per subscriber (service per SAP) – only expanded SLA mode
The example in this chapter is based on subscriber hosts and the subscriber session concept is disabled.
A few tens of egress queues are allocated for internal use, so are unavailable for subscribers.
The number of subscribers per HSQ IOM in expanded SLA mode depends on the number of available HS primary shapers (16k in total). Because some of the HS primary shapers may be used internally, it is recommended that the exact number of free resources on an HSQ card is periodically checked with the tools dump system-resources command.
Because each SAP under the same subscriber requires its own SPI, this model is not supported in single SLA mode.
HSQ IOM supports an enhanced egress QoS architecture to provide scalable network, service, and subscriber QoS. At ingress, the HSQ supports regular FP3 QoS with a high ingress policer scaling.
The emphasis in this chapter is on egress shaping and scheduling in the ESM context. Buffer pool management on HSQ is not described in this chapter. Generic ESM concepts are described in other chapters, but for the sake of completeness, a basic ESM configuration as it applies to the example in this chapter is outlined in Appendix A — Generic ESM configuration.
This chapter is conceptually divided into two parts:
The focus of the first part is on the (egress) QoS configuration for three subscribers, named ‟sub-1”, ‟sub-2”, and ‟sub-3”.
The second part focuses on examining HSQ traffic management capabilities in the ESM context by observing output traffic patterns in a congested system.
Topics related to the generic operation of HSQ IOM and ESM that are not directly described in this chapter are included in the following:
Configuring expanded SLA mode in ESM is in the chapter High Scale QoS IOM in ESM Context: Expanded SLA Mode.
Configuring HSQ in the service context is in the chapter High Scale QoS IOM: QoS, Service and Network Configuration.
Generic ESM concepts are described in the 7450 ESS, 7750 SR, and VSR Triple Play Service Delivery Architecture Guide and in numerous TPSDA chapters in the ACG.
Test environment
Test environment example shows the test environment example with three subscribers in single SLA mode, each with a single DHCPv4 host setup in the BNG.
DHCPv4 traffic (simulated subscriber hosts) is initiated from port 103/1 on a traffic generator. Subscribers are authenticated via local user database (LUDB) and instantiated on managed SAPs (MSAPs) in the IES 3 service. Subscriber IP addresses are assigned statically via LUDB.
To explore traffic management capabilities under congestion, a number of traffic streams are generated in the downstream direction, from port 203/1 on the same traffic generator toward the subscribers on port 103/1.
The QoS hierarchy associated with the subscribers is shown in QoS hierarchy in single SLA mode. At a high level, this hierarchy can be described as follows:
Each of the three subscribers (sub-1, sub-2, and sub-3) is associated with an HSQ queue group. The HSQ queue group always comprises eight queues. However, not all the queues are required to be used by the subscriber. In this example, only six queues are used by each subscriber, while the two remaining queues, although allocated to the subscriber, remain unused.
All three subscribers share the same HS attachment policy. This means that the mapping between the queues, scheduling classes (SC), and WRR groups is the same for all three subscribers.
Queues 1 and 2 are attached to WRR group 1 at the local HSQ queue group level and served in 4:1 ratio. WRR group 1 is attached to the lowest scheduling class 1, whereas queues 3, 4, 5, and 6 are directly attached to scheduling classes 3, 4, 5, and 6, respectively.
Each subscriber is rate limited at the aggregate level.
Subscribers are mapped to two HS secondary shapers. Limiting the aggregate rates of these shapers will ensure that the bandwidth capacity of the corresponding access nodes in the network is not overrun.
Mapping between the subscribers and their HS secondary shapers is achieved via outer VLANs. Sub-1 and sub-2 on SAPs with the outer VLAN 1 are mapped to HS secondary shaper 1 while sub-3 on outer VLAN 3 is mapped to HS secondary shaper 2.
The two HS secondary shapers are associated with the HS port scheduler that has its own aggregate rate limit set.
At the port level, scheduling class 6 is rate limited while scheduling classes 4 and 5 are collapsed into a WRR group, which is rate limited.
Configuration
The configuration section is split into two parts:
HSQ-specific ESM configuration
The generic ESM configuration used in this example is provided in Appendix A — Generic ESM configuration, for the sake of completeness. This section focuses only on HSQ-specific ESM configuration.
Each of the three subscribers have their own sub-profile, which will determine the HS SLA mode in which the subscriber operates, as well as the aggregate rate limit (in kb/s) for each subscriber:
configure
subscriber-mgmt
sub-profile "sub-prof-1" create
hs-sla-mode single
egress
hs-agg-rate-limit 50000
exit
exit
sub-profile "sub-prof-2" create
hs-sla-mode single
egress
hs-agg-rate-limit 75000
exit
exit
sub-profile "sub-prof-3" create
hs-sla-mode single
egress
hs-agg-rate-limit 100000
exit
exit
Each subscriber has its own SPI that references the QoS SAP policy on egress (in this case, QoS SAP egress policies 10, 20, and 30). The egress QoS SAP policy defines the QoS characteristics at the local subscriber level (or HSQ queue group level), such as:
traffic classification
queue/WRR rates and weights
mapping between the queues, WRR groups, and scheduling classes by referencing the HS attachment policy
configure
subscriber-mgmt
sla-profile "sla-1" create
egress
qos 10
exit
exit
exit
sla-profile "sla-2" create
egress
qos 20
exit
exit
exit
sla-profile "sla-3" create
egress
qos 30
exit
exit
exit
MSAP policy is a mandatory configuration for dynamically created SAPs (Managed SAP or MSAPs) where, in the context of HSQ, a few key parameters are provisioned:
The def-inter-dest-id use-top-q command associates subscribers with the HS secondary shaper by matching the outer VLAN ID of the subscriber SAP to the HS secondary shaper name. For example, a subscriber on SAP 3/1/1:1.2 will be mapped to HS secondary shaper 1:
configure port 3/1/1 ethernet mode access encap-type qinq egress hs-scheduler-policy "hs1" hs-secondary-shaper "1" create aggregate rate 150000 exit exit exit exit exitThe qos 1 service-queuing command disables shared queueing on ingress because ingress shared queuing is not supported on HSQ. With shared queuing disabled and service queuing enabled, every configured subscriber or SAP queue on ingress is potentially allocated multiple times, which leads to inefficient use of resources. Every configured queue on ingress is multiplied by the number of possible destination forwarding complexes. For this reason, Nokia recommends that policers are used on ingress.
The profiled-traffic-only command provides a resource optimization that removes queues from the subscriber SAP. By default, one ingress and one egress queue is instantiated per subscriber SAP due to a default QoS SAP policy "1" that is applied to a SAP. In most cases, these SAP queues remain unused because each subscriber uses its own sets of separate subscriber queues that are allocated per SPI. Therefore, the SAP queues can be safely removed.
Note:The default SAP egress policy can be replaced with another, non-default SAP egress policy. However, the SAP egress policy cannot be removed from a SAP.
A SAP queue in the ESM context on HSQ is used when multicast per SAP replication mode for the subscriber is enabled. In multicast per SAP replication mode, a single copy of each multicast stream is sent for all hosts of the subscriber via a SAP queue. If the SAP queue is removed, multicast traffic will flow via failover queues or via a queue group that must be explicitly configured.
For example, if there are 50k subscribers on an HSQ IOM, with each subscriber on its own SAP, then removing the default SAP queues:
preserves 50k queues on ingress. These ingress queues are allocated from the 128k ingress queue pool and are now available for ingress subscriber queuing.
Removing the SAP queues on ingress becomes even more relevant on HSQ, which does not support shared queuing on ingress. Non-shared queuing on ingress means that the number of ingress queues allocated per each SAP would increase proportionally with the number of destination forwarding complexes in the system.
preserves 50k queues on egress from the 768k egress queue pool. Because each subscriber requires 8 queues on egress, these 50k egress queues can be used for an additional ~6k subscribers.
SAP queue removal works only for subscribers in 1:1 model (subscriber per SAP) and where the number of subscribers per SAP is limited to 1. This requires one additional command: no multi-sub-sap (or alternatively multi-sub-sap limit 1).
The following command can be used to verify that the SAP queues have been removed:
show service id <srvc-id> sap <sap-id> statsThe resulting msap-policy configuration is as follows:
configure
subscriber-mgmt
msap-policy "msaps" create
sub-sla-mgmt
def-inter-dest-id use-top-q
sub-ident-policy "sub_ident_pol"
no multi-sub-sap
single-sub-parameters
profiled-traffic-only
exit
exit
ies-vprn-only-sap-parameters
ingress
qos 1 service-queuing
exit
exit
exit
HSQ-specific QoS configuration
At the subscriber level (or HSQ queue group level), the HS attachment policy defines mapping between the queues, WRR groups, and scheduling classes. In this example, queues 1 and 2 are mapped to a WRR group 1, which is mapped into scheduling class 1. Queues 3 to 6 are mapped to respective scheduling classes (SC 3 to 6). Queues 7 and 8, although allocated, are unattached, so will drop any traffic that they receive. A maximum of two WRR groups are supported at the HSQ queue group level and in this example, only WRR 1 is used.
The low-burst-max-class parameter is set to 3, which ensures that the 3 lower scheduling classes (1 to 3) are stopped being served first if HSQ queue group aggregate congestion occurs. The HSQ queue group aggregate shaper has two burst tolerance thresholds: when the first threshold is reached, scheduling classes 1 to 3 will be removed from the service list, followed by the higher level scheduling classes (4 to 6) being removed when the second threshold is reached. This designation of scheduling classes in two tiers ensures lower latency for traffic associated with higher scheduling classes during short-lived congestion periods.
The following defined HS attachment policy is applied to the subscriber through the SAP egress policy referenced in the SLA profile. In this example, all three subscribers use the same HS attachment policy:
configure
qos
hs-attachment-policy "hs-attach-1-1" create
low-burst-max-class 3
queue 1 wrr-group 1
queue 2 wrr-group 1
queue 3 sched-class 3
queue 4 sched-class 4
queue 5 sched-class 5
queue 6 sched-class 6
queue 7 unattached
queue 8 unattached
wrr-group 1 sched-class 1
wrr-group 2 unattached
exit
exit
Besides referencing the HS attachment policy, the SAP egress policy defines traffic classification, as well as characteristics of queues and WRR groups at the subscriber level. In this example, the SAP egress policy 10 is associated with sub-1. The queues 1 and 4 are in the context of WRR group 1 serviced in the ratio 1:4.
Although the SAP egress policy syntax implies that such policy is applied per SAP, in the ESM context this policy is instantiated via the SLA profile; therefore, the queue/policer instantiations are performed per SPI and not per SAP.
The aggregate rate of WRR 1 is set to 20 Mb/s. Mapping of forwarding classes to queues is self-explanatory. Similarly, SAP egress policies are applied to sub-2 (policy 20) and sub-3 (policy 30). The only difference between the QoS SAP egress policies for the subscribers is the rate of the WRR group 1, which is 40 Mb/s for sub-2 and 60 Mb/s for sub-3.
The SAP egress policy is applied in the SLA profile for the subscriber. The WRR group rate (along with other QoS parameters) can be dynamically overridden via RADIUS/Diameter during authentication or while the host/session is online. This functionality would allow having only one SAP egress policy configured where parameters can be dynamically overridden, when the policy is applied to the subscriber host or session.
configure
qos
sap-egress 10 create
hs-attachment-policy "hs-attach-1-1"
queue 1 create
hs-wrr-weight 1
exit
queue 2 create
hs-wrr-weight 4
exit
queue 3 create
exit
queue 4 create
exit
queue 5 create
exit
queue 6 create
exit
hs-wrr-group 1
rate 20000
exit
fc af create
queue 3
exit
fc be create
queue 1
exit
fc h1 create
queue 5
exit
fc h2 create
queue 6
exit
fc l1 create
queue 4
exit
fc l2 create
queue 2
exit
dscp af12 fc "af"
dscp be fc "be"
dscp af22 fc "h1"
dscp ef fc "h2"
dscp af21 fc "l1"
dscp af11 fc "l2"
exit
The next hierarchy level in the chain (up from the subscriber level) is performed by the two HS secondary shapers that are directly configured in the egress context of the port. The names of the two HS secondary shapers correspond to the outer VLANs on the subscriber SAPs. The network represents the HS secondary shapers as the access nodes downstream from BNG.
The HS secondary shapers "1" and "2" are configured with the aggregate rates of 120 Mb/s and 150 Mb/s, respectively. The rates are configured in kb/s. Similarly, as at the subscriber level, the low-burst-max-class 3 command maps all scheduling classes at or below level 3 to the low burst tolerance threshold, while all scheduling classes above level 3 are mapped to the high burst tolerance threshold at the HS secondary shaper level. Therefore, in case of congestion, objects associated with scheduling classes 3 and below will be removed from the service list before the objects associated with scheduling classes 4 and above.
configure
port 3/1/1
ethernet
mode access
encap-type qinq
egress
hs-scheduler-policy "hs1"
hs-secondary-shaper "1" create
aggregate
rate 120000
low-burst-max-class 3
exit
exit
hs-secondary-shaper "2" create
aggregate
rate 150000
low-burst-max-class 3
exit
exit
exit
exit
no shutdown
exit
The last configuration block in the scheduling hierarchy is an HS port scheduler, which is associated with the port via the command hs-scheduler-policy "hs1" in the preceding CLI code.
The HS port scheduler characteristics in this example are defined as follows:
The maximum rate is set to 200 Mb/s.
Scheduling classes 4 and 5 are collapsed into a single scheduling priority (5) and they are served in a 1:2 ratio. This is performed via WRR group 1.
This collapsing of scheduling classes 4 and 5 occurs at the port level, whereas scheduling classes 1 and 2 are collapsed at the subscriber level (HSQ queue group level).
The WRR 1 at the port level is rate limited to 60 Mb/s.
The highest priority (6) scheduling class is rate limited to 20 Mb/s at the port level.
configure
qos
hs-scheduler-policy "hs1" create
max-rate 200
group 1 rate 60
scheduling-class 4 group 1 weight 10
scheduling-class 5 group 1 weight 20
scheduling-class 6 rate 20
exit
exit
The delta between the low and high burst tolerance thresholds at the subscriber and HS secondary shaper levels can be adjusted with the hs-fixed-high-thresh-delta command that is configured at the card level. In this example, the difference between the thresholds is set to 12 kbytes. The default value is 4000 bytes.
The lower threshold for the buckets is calculated automatically by the system, based on many internal inputs (clock frequency of rate timer, shaper type, heuristics, MDA type, and so on).
configure
card 3
card-type iom4-e-hs
mda 1
mda-type me10-10gb-sfp+
no shutdown
exit
fp 1
egress
hs-fixed-high-thresh-delta 12000
exit
exit
no shutdown
Operational commands
Operational commands are used to troubleshoot the system and monitor its operational state. The focus of this section will be on show, clear, and tools dump commands. The debug commands are omitted because there are no debug commands related to QoS on HSQ IOM and the debug commands related to ESM are described in other chapters. Also, there are no log events related to QoS on HSQ IOM.
Show commands
show commands in this section are divided into two groups:
show commands that display association between ESM and QoS objects where all displayed information is static and does not change autonomously over time.
show commands that display QoS hierarchy with the running rates (where the state is changing autonomously).
show commands have filters (or CLI parameters) that can be used to control the amount of the output information. This section will provide a few show command examples; it is left to the user to explore all the options available for a particular show command.
Subscriber management related show commands
Examining subscriber associations with QoS objects should begin with the show service active-subscribers command. The output of this command provides information about the subscriber context and the output can vary in detail depending on the options with which this command is run. Besides the subscriber name, underlying subscriber SAPs, and SLA/sub-profile names, the following information is provided:
SLA mode in which the subscriber operates on HSQ (single versus expanded SLA mode)
Aggregate rate of the subscriber
Ingress and egress QoS policies
Subscriber association with an HS secondary shaper and the inter dest id string that is used to make this association
Ingress queue/policer statistics
Egress queues statistics
For brevity, only the information for subscriber sub-1 is shown in the following output:
A:PE-1# show service active-subscribers subscriber "sub-1" detail
===============================================================================
Active Subscribers
===============================================================================
-------------------------------------------------------------------------------
Subscriber sub-1 (sub-prof-1)
-------------------------------------------------------------------------------
I. Policer Ctrl. : N/A
E. Policer Ctrl. : N/A
I. vport-hashing : Disabled
I. sec-sh-hashing: Disabled
Q Frame-Based Ac*: Disabled
Acct. Policy : N/A
Collect Stats : Disabled
ANCP Pol. : N/A
Accu-stats-pol : (Not Specified)
HostTrk Pol. : N/A
IGMP Policy : N/A
MLD Policy : N/A
PIM Policy : N/A
Sub. MCAC Policy : N/A
NAT Policy : N/A
Firewall Policy : N/A
UPnP Policy : N/A
NAT Prefix List : N/A
Def. Encap Offset: none
Encap Offset Mode: none
Avg Frame Size : N/A
Vol stats type : full
Preference : 5
LAG hash class : 1
LAG hash weight : 1
Sub. ANCP-String : "sub-1"
Sub. Int Dest Id : "1"
Igmp Rate Adj : N/A
RADIUS Rate-Limit: N/A
Oper-Rate-Limit : 50000
-------------------------------------------------------------------------------
Radius Accounting
-------------------------------------------------------------------------------
Policy : N/A
Session Opti.Stop: False
-------------------------------------------------------------------------------
HS
-------------------------------------------------------------------------------
SLA-mode : single E Agg Rate Limit : 50000
Hs Second Shaper : "1"
* indicates that the corresponding row element may have been truncated.
-------------------------------------------------------------------------------
(1) SLA Profile Instance
- sap:[3/1/1:1.1] (IES 3 - group-int-1)
- sla:sla-1
-------------------------------------------------------------------------------
Description : (Not Specified)
Host Limits : No Limit
Egr Sched-Policy : N/A
Ingress Qos-Policy : 1 Egress Qos-Policy : 10
Ingress Queuing Type : Service-queuing (Not Applicable to Policer)
Ingr IP Fltr-Id : N/A Egr IP Fltr-Id : N/A
Ingr IPv6 Fltr-Id : N/A Egr IPv6 Fltr-Id : N/A
Ingress Report-Rate : Maximum
Egress Report-Rate : Maximum
Egress Remarking : from Sap Qos
Credit Control Pol. : N/A
Category Map : (Not Specified)
Use ing L2TP DSCP : false
Hs-Agg-Rate-Limit : Maximum
Egress HS Q stat mode: no-override
Hs-Oper-Rate-Limit : Maximum
Egr hqos mgmt status : disabled
-------------------------------------------------------------------------------
-------------------------------------------------------------------------------
IP Address
MAC Address Session Origin Svc Fwd
-------------------------------------------------------------------------------
10.10.1.1
00:00:64:01:01:01 N/A DHCP 3 Y
-------------------------------------------------------------------------------
-------------------------------------------------------------------------------
SLA Profile Instance statistics
-------------------------------------------------------------------------------
Packets Octets
Off. HiPrio : 0 0
Off. LowPrio : 0 0
Off. Uncolor : 0 0
Off. Managed : 0 0
Queueing Stats (Ingress QoS Policy 1)
Dro. HiPrio : 0 0
Dro. LowPrio : 0 0
For. InProf : 0 0
For. OutProf : 0 0
Queueing Stats (Egress QoS Policy 10)
Dro. In/InplusProf : 88117477 88822416816
Dro. Out/ExcProf : 68563861 69112371888
For. In/InplusProf : 41604295 41937129360
For. Out/ExcProf : 87102263 87799081104
-------------------------------------------------------------------------------
SLA Profile Instance per Queue statistics
-------------------------------------------------------------------------------
Packets Octets
Ingress Queue 1 (Unicast) (Priority)
Off. HiPrio : 0 0
Off. LowPrio : 0 0
Dro. HiPrio : 0 0
Dro. LowPrio : 0 0
For. InProf : 0 0
For. OutProf : 0 0
Egress Queue 1
Dro. In/InplusProf : 0 0
Dro. Out/ExcProf : 62846249 63349018992
For. In/InplusProf : 0 0
For. Out/ExcProf : 2014638 2030755104
Egress Queue 2
Dro. In/InplusProf : 56805196 57259637568
Dro. Out/ExcProf : 0 0
For. In/InplusProf : 8055689 8120134512
For. Out/ExcProf : 0 0
Egress Queue 3
Dro. In/InplusProf : 0 0
Dro. Out/ExcProf : 256284 258334272
For. In/InplusProf : 0 0
For. Out/ExcProf : 51632424 52045483392
Egress Queue 4
Dro. In/InplusProf : 22176682 22354095456
Dro. Out/ExcProf : 0 0
For. In/InplusProf : 16739851 16873769808
For. Out/ExcProf : 0 0
Egress Queue 5
Dro. In/InplusProf : 0 0
Dro. Out/ExcProf : 5461328 5505018624
For. In/InplusProf : 0 0
For. Out/ExcProf : 33455201 33722842608
Egress Queue 6
Dro. In/InplusProf : 9135599 9208683792
Dro. Out/ExcProf : 0 0
For. In/InplusProf : 16808755 16943225040
For. Out/ExcProf : 0 0
To reveal the subscriber hierarchy in a terse form with respect to the sub/SLA-profiles and the SAP, the following command can be run:
A:PE-1# show service active-subscribers hierarchy
===============================================================================
Active Subscribers Hierarchy
===============================================================================
-- sub-1 (sub-prof-1)
|
+-- sap:[3/1/1:1.1] - sla:sla-1
|
+-- 10.10.1.1 - mac:00:00:64:01:01:01 - DHCP - svc:3
-- sub-2 (sub-prof-2)
|
+-- sap:[3/1/1:1.2] - sla:sla-2
|
+-- 10.10.1.2 - mac:00:00:64:01:01:02 - DHCP - svc:3
-- sub-3 (sub-prof-3)
|
+-- sap:[3/1/1:2.1] - sla:sla-3
|
+-- 10.10.1.3 - mac:00:00:64:01:01:03 - DHCP - svc:3
-------------------------------------------------------------------------------
Number of active subscribers : 3
Flags: (N) = the host or the managed route is in non-forwarding state
===============================================================================
The following SAP related show command confirms that the SAP queues are removed from the underlying subscriber SAP (under the stats section at the end of the output). This was ensured by configuring the profiled-traffic-only command in the MSAP policy, with the purpose of reducing the queue consumption on ingress and egress.
A:PE-1# show service id 3 sap 3/1/1:1.1 detail
===============================================================================
Service Access Points(SAP)
===============================================================================
Service Id : 3
SAP : 3/1/1:1.1 Encap : qinq
QinQ Dot1p : Default
Description : Managed SAP - Capture Svc 10 3/1/1:*.*
Admin State : Up Oper State : Up
Flags : None
Multi Svc Site : None
Last Status Change : 09/08/2017 15:16:20
Last Mgmt Change : 09/11/2017 14:22:55
Sub Type : managed
Capture Service Id : 10 Capture SAP : 3/1/1:*.*
MSAP Policy : msaps
Idle : no Sticky : no
Dot1Q Ethertype : 0x8100 QinQ Ethertype : 0x8100
Split Horizon Group: (Not Specified)
Admin MTU : 1522 Oper MTU : 1522
Ingr IP Fltr-Id : n/a Egr IP Fltr-Id : n/a
Ingr Mac Fltr-Id : n/a Egr Mac Fltr-Id : n/a
Ingr IPv6 Fltr-Id : n/a Egr IPv6 Fltr-Id : n/a
qinq-pbit-marking : both
Egr Agg Rate Limit: max
Q Frame-Based Acct : Disabled Limit Unused BW : Disabled
Acct. Pol : None Collect Stats : Disabled
Anti Spoofing : Ip-Mac Dynamic Hosts : Enabled
Avl Static Hosts : 0 Tot Static Hosts : 0
Calling-Station-Id : n/a
Application Profile: None
Transit Policy : None
AARP Id : None
Oper Group : (none) Monitor Oper Grp : (none)
Host Lockout Plcy : n/a
Lag Link Map Prof : (none)
Bandwidth : Not-Applicable
Oper DCpu Prot Pol*: _default-access-policy
-------------------------------------------------------------------------------
ETH-CFM SAP specifics
-------------------------------------------------------------------------------
Tunnel Faults : accept AIS : Disabled
MC Prop-Hold-Timer : n/a
Squelch Levels : None
Collect Lmm Stats : Disabled
LMM FC Stats : None
LMM FC In Prof : None
-------------------------------------------------------------------------------
QOS
-------------------------------------------------------------------------------
Ingress qos-policy : 1 Egress qos-policy : 1
Ingress FP QGrp : (none) Egress Port QGrp : (none)
Ing FP QGrp Inst : (none) Egr Port QGrp Inst: (none)
Shared Q plcy : n/a Multipoint shared : Disabled
I. Sched Pol : (Not Specified)
E. Sched Pol : (Not Specified)
I. Policer Ctl Pol : (Not Specified)
E. Policer Ctl Pol : (Not Specified)
E. HS Sec. Shaper : (Not Specified)
I. QGrp Redir. List: (Not Specified)
E. QGrp Redir. List: (Not Specified)
-------------------------------------------------------------------------------
Subscriber Management
-------------------------------------------------------------------------------
Admin State : Up MAC DA Hashing : False
Def Sub-Id : Use auto-sub-id
Def Sub-Profile : None
Def SLA-Profile : None
Def Inter-Dest-Id : (Use top-q-tag)
Def App-Profile : None
Sub-Ident-Policy : sub_ident_pol
Subscriber Limit : 1
Single-Sub-Parameters
Prof Traffic Only : True
Non-Sub-Traffic : N/A
Static host management
MAC learn options : N/A
-------------------------------------------------------------------------------
Sap Statistics
-------------------------------------------------------------------------------
Last Cleared Time : N/A
Packets Octets
CPM Ingress : 3 443
Forwarding Engine Stats
Dropped : 0 0
Received Valid : 0 0
Off. HiPrio : 0 0
Off. LowPrio : 0 0
Off. Uncolor : 0 0
Off. Managed : 0 0
Queueing Stats(Ingress QoS Policy 1)
Dro. HiPrio : 0 0
Dro. LowPrio : 0 0
For. InProf : 0 0
For. OutProf : 0 0
Queueing Stats(Egress QoS Policy 1)
Dro. In/InplusProf : 0 0
Dro. Out/ExcProf : 0 0
For. In/InplusProf : 0 0
For. Out/ExcProf : 0 0
-------------------------------------------------------------------------------
Sap per Queue stats
-------------------------------------------------------------------------------
Packets Octets
No entries found
===============================================================================
* indicates that the corresponding row element may have been truncated.
A:PE-1#
QoS-related show commands in ESM context
Examination of the subscriber QoS hierarchy on HSQ can start at the subscriber (HSQ queue group and SAP) level and gradually move through the HS secondary shaper, and finally the port level. The output of the show commands should confirm that the subscriber is associated with the QoS object as intended by the configuration.
For example, the following output shows that HS attachment policy "hs-attach-1-1" is associated with QoS SAP egress policy 10. It was determined previously that QoS SAP egress policy 10 is associated with subscribersub-1. In this case, a two-step process was necessary to track the association between the subscriber and the HS attachment policy.
A:PE-1# show qos sap-egress 10 association
===============================================================================
QoS Sap Egress
===============================================================================
-------------------------------------------------------------------------------
Sap Egress Policy (10)
-------------------------------------------------------------------------------
Policy-id : 10 Scope : Template
Ethernet-ctag : False Parent-loc : default
Name : (Not Specified)
Description : (Not Specified)
Policy Active : True Plcrs HQoS Managed : False
Post Plcr Mapping Policy: (Not Specified)
HS Attachment Policy : hs-attach-1-1
-------------------------------------------------------------------------------
Dynamic Configuration Information
-------------------------------------------------------------------------------
PccRule Insert Point : n/a DynPlcr Insert Point : n/a
CBS : Def MBS : Def
Parent : (Not Specified)
Level : 1 Weight : 1
Packet Byte Offset : 0
Stat Mode : minimal
-------------------------------------------------------------------------------
-------------------------------------------------------------------------------
Associations
-------------------------------------------------------------------------------
SLA Profiles :
- sla-1
-------------------------------------------------------------------------------
HSMDA Associations
-------------------------------------------------------------------------------
No Associations Found.
===============================================================================
The output from the HS attachment policy reflects the QoS configuration state at the subscriber level that is shown in QoS hierarchy in single SLA mode:
Queues 1 and 2 are attached to WRR 1.
Queues 3 to 6 are directly attached to the corresponding scheduling classes (3 to 6).
Queues 7 and 8 are unattached.
WRR 2 is unattached.
HS attachment policy "hs-attach-1" is associated with QoS SAP egress policies 10, 20, and 30 that correspond to sub-1, sub-2, and sub-3.
A:PE-1# show qos hs-attachment-policy "hs-attach-1-1" detail
===============================================================================
HS Attachment Policy Information
===============================================================================
Policy Name : hs-attach-1-1
Description : (Not Specified)
Low Burst Max Class : 3
-------------------------------------------------------------------------------
Queue Scheduling Class WRR Group
-------------------------------------------------------------------------------
1 (Not-Applicable) 1
2 (Not-Applicable) 1
3 3 (Not-Applicable)
4 4 (Not-Applicable)
5 5 (Not-Applicable)
6 6 (Not-Applicable)
7 unattached unattached
8 unattached unattached
-------------------------------------------------------------------------------
WRR Group Scheduling Class
-------------------------------------------------------------------------------
1 1
2 unattached
-------------------------------------------------------------------------------
Associations
-------------------------------------------------------------------------------
Network-Queue Policy
-------------------------------------------------------------------------------
No Matching Entries
Sap-Egress Policy
-------------------------------------------------------------------------------
10
20
30
Egress Queue-Group Templates
-------------------------------------------------------------------------------
No Matching Entries
-------------------------------------------------------------------------------
Association between the HS secondary shaper and the subscribers can be verified with the following command. The HS secondary shaper is allocated per port (or LAG). The two subscribers (sub-1 and sub-2) are instantiated on SAPs with the outer VLAN tag 1; consequently, they are both associated with HS secondary shaper 1. The HS secondary shaper 1 is rate limited to 120 Mb/s while its scheduling classes are left open (max rate).
A:PE-1# show port 3/1/1 hs-secondary-shaper "1" associations
===============================================================================
Ethernet Port 3/1/1 Egress HS Secondary Shaper Information
===============================================================================
Policy Name : 1
Description : (Not Specified)
Rate : 120000 Kbps
Low Burst Max Class: 3
-------------------------------------------------------------------------------
Class Rate
-------------------------------------------------------------------------------
1 max
2 max
3 max
4 max
5 max
6 max
-------------------------------------------------------------------------------
-------------------------------------------------------------------------------
Service Associations
-------------------------------------------------------------------------------
Service ID Service Type SAP
-------------------------------------------------------------------------------
No Service Associations Found.
-------------------------------------------------------------------------------
-------------------------------------------------------------------------------
Subscriber Associations
-------------------------------------------------------------------------------
Subscriber ID
-------------------------------------------------------------------------------
sub-1
sub-2
-------------------------------------------------------------------------------
Number of subscriber associations : 2
-------------------------------------------------------------------------------
The port scheduler information can be obtained with the following command. This command is run in the QoS context (as opposed to being run in the port context, which was the case for HS secondary schedulers). The reason for this is that the HS secondary scheduler is configured directly under the port, while the HS port scheduler is configured in an HS scheduler policy (in QoS context), which is then applied to a port.
A:PE-1# show qos hs-scheduler-policy "hs1" detail
===============================================================================
HS Scheduler Policy Information
===============================================================================
Policy Name : hs1
Description : (Not Specified)
Max Rate : 200 Mbps
-------------------------------------------------------------------------------
Scheduling Class Rate Group Weight in Group
-------------------------------------------------------------------------------
1 max 0 1
2 max 0 1
3 max 0 1
4 max 1 10
5 max 1 20
6 20 Mbps 0 1
-------------------------------------------------------------------------------
Group Rate
-------------------------------------------------------------------------------
1 60 Mbps
-------------------------------------------------------------------------------
Port Ethernet Egress Associations
-------------------------------------------------------------------------------
3/1/1
-------------------------------------------------------------------------------
===============================================================================
Show commands with dynamically changing information
The following commands show the QoS hierarchy with the measured rates of the objects (queues, WRR groups, scheduling classes, secondary shapers, and port shapers) and the queue buffer depths in the QoS hierarchy. The command output in this section is based on the scenario described in the Traffic management on HSQ section.
The following subscriber hierarchy shows the rates per scheduling priority (and consequently, scheduling class), starting at the queue level and moving up toward the port level. For example, scheduling priority 1 starts with the rates of two subscriber queues (1 and 2) that are mapped to the WRR group. Scheduling priority 1 then moves to the rate at the HS secondary shaper level (the summed rate of all the entities at scheduling priority 1 at the HS secondary shaper level), ending with the rate of the scheduling class 1 at the port level. The aggregate rate of the HS secondary shaper and the port in the subscriber hierarchy are also provided.
A:PE-1# show qos hs-scheduler-hierarchy subscriber "sub-1" egress
===============================================================================
Hs Scheduler Hierarchy Information
===============================================================================
PortId : 3/1/1
SAP : [3/1/1:1.1]
SLA Profile : sla-1
Hs Sched Policy Name : hs1
Port Max-Rate : 200 Mbps
Hs-Sec-Shaper:1 Agg-Rate : 120017 Kbps
Scheduler Priority 6
Scheduler Class 6 Rate : 19 Mbps
Hs-Sec-Shaper:1 Class 6 Rate : 13349 Kbps
Queue : 6 Rate : 6528 Kbps
Scheduler Priority 5 Group 1
Scheduler Class 5 Rate : 40 Mbps Weight : 20
Hs-Sec-Shaper:1 Class 5 Rate : 26699 Kbps
Queue : 5 Rate : 13072 Kbps
Scheduler Class 4 Rate : 19 Mbps Weight : 10
Hs-Sec-Shaper:1 Class 4 Rate : 13300 Kbps
Queue : 4 Rate : 6528 Kbps
Scheduler Priority 3
Scheduler Class 3 Rate : 61 Mbps
Hs-Sec-Shaper:1 Class 3 Rate : 40800 Kbps
Queue : 3 Rate : 20008 Kbps
Scheduler Priority 2
Scheduler Class 2 Rate : 0 Mbps
Hs-Sec-Shaper:1 Class 2 Rate : 0 Kbps
Scheduler Priority 1
Scheduler Class 1 Rate : 58 Mbps
Hs-Sec-Shaper:1 Class 1 Rate : 25867 Kbps
Queue : 1 Group : 1 Rate : 752 Kbps
Queue : 2 Group : 1 Rate : 3104 Kbps
===============================================================================
The following command provides the measured rates at the HS secondary shaper and port levels:
A:PE-1# show qos hs-scheduler-hierarchy port 3/1/1 hs-secondary-shapers
===============================================================================
Hs Scheduler Hierarchy Information
===============================================================================
Hs Sched Policy Name : hs1
Port Max-Rate : 200 Mbps
Scheduler Priority 6
Scheduler Class 6 Rate : 20 Mbps
Scheduler Priority 5 Group 1
Scheduler Class 5 Rate : 40 Mbps Weight : 20
Scheduler Class 4 Rate : 19 Mbps Weight : 10
Scheduler Priority 3
Scheduler Class 3 Rate : 61 Mbps
Scheduler Priority 2
Scheduler Class 2 Rate : 0 Mbps
Scheduler Priority 1
Scheduler Class 1 Rate : 58 Mbps
-------------------------------------------------------------------------------
HS Secondary Shaper Rates
-------------------------------------------------------------------------------
Hs-Sec-Shaper:1 Agg-Rate : 119519 Kbps
Class 6 Rate : 13268 Kbps
Class 5 Rate : 26544 Kbps
Class 4 Rate : 13284 Kbps
Class 3 Rate : 40661 Kbps
Class 2 Rate : 0 Kbps
Class 1 Rate : 25761 Kbps
Hs-Sec-Shaper:2 Agg-Rate : 79764 Kbps
Class 6 Rate : 6642 Kbps
Class 5 Rate : 13300 Kbps
Class 4 Rate : 6625 Kbps
Class 3 Rate : 20342 Kbps
Class 2 Rate : 0 Kbps
Class 1 Rate : 32852 Kbps
Hs-Sec-Shaper:default Agg-Rate : 0 Kbps
Class 6 Rate : 0 Kbps
Class 5 Rate : 0 Kbps
Class 4 Rate : 0 Kbps
Class 3 Rate : 0 Kbps
Class 2 Rate : 0 Kbps
Class 1 Rate : 0 Kbps
-------------------------------------------------------------------------------
Another important parameter to monitor is the depth of the queues. This provides information about the degree of congestion at the queue level. The buffer space per queue is allocated automatically by the system and, in the following case, the buffers are rather large. Deep buffering causes longer delays. To avoid this, the queue buffers can be adjusted by the mbs command under the queue definition in the QoS SAP egress policy. For example, 15 kbytes would accommodate roughly fifteen 1000 byte packets in a buffer queue.
*A:PE-1>config>qos>sap-egress# info
----------------------------------------------
queue 1 create
mbs 15 kilobytes
The following output shows that, except for queue 3, all queues have their buffers fully used. Because there is no congestion on queue 3, its buffer is unused.
A:PE-1# show hs-pools port 3/1/1 egress subscriber "sub-1" | match "Queue Information" pre-lines 1 post-lines 100
-------------------------------------------------------------------------------
Queue Information
-------------------------------------------------------------------------------
Queue Name : Sub=sub-1:sla-1 3->3/1/1:1.1->1
FC Map : be ef nc
Admin PIR : 20000 Oper PIR : 0
Admin MBS : 64 KB Oper MBS : 64 KB
HS Wrr Group : 1
HS Wrr Class Weight: 1 HS Wrr Weight : 1
Depth : 58 KB
HS Class : 1 HS Alt Port Class Pool : No
HS Slope Policy : _tmnx_hs_default
Queue Name : Sub=sub-1:sla-1 3->3/1/1:1.1->2
FC Map : l2
Admin PIR : 20000 Oper PIR : 0
Admin MBS : 64 KB Oper MBS : 64 KB
HS Wrr Group : 1
HS Wrr Class Weight: 1 HS Wrr Weight : 4
Depth : 58 KB
HS Class : 1 HS Alt Port Class Pool : No
HS Slope Policy : _tmnx_hs_default
Queue Name : Sub=sub-1:sla-1 3->3/1/1:1.1->3
FC Map : af
Admin PIR : Max Oper PIR : Max
Admin MBS : 262500 B Oper MBS : 262656 B
HS Wrr Group : (not-applicable)
HS Wrr Class Weight: 1 HS Wrr Weight : 0
Depth : 2 KB
HS Class : 3 HS Alt Port Class Pool : No
HS Slope Policy : _tmnx_hs_default
Queue Name : Sub=sub-1:sla-1 3->3/1/1:1.1->4
FC Map : l1
Admin PIR : Max Oper PIR : Max
Admin MBS : 262500 B Oper MBS : 262656 B
HS Wrr Group : (not-applicable)
HS Wrr Class Weight: 1 HS Wrr Weight : 0
Depth : 233 KB
HS Class : 4 HS Alt Port Class Pool : No
HS Slope Policy : _tmnx_hs_default
Queue Name : Sub=sub-1:sla-1 3->3/1/1:1.1->5
FC Map : h1
Admin PIR : Max Oper PIR : Max
Admin MBS : 262500 B Oper MBS : 262656 B
HS Wrr Group : (not-applicable)
HS Wrr Class Weight: 1 HS Wrr Weight : 0
Depth : 233 KB
HS Class : 5 HS Alt Port Class Pool : No
HS Slope Policy : _tmnx_hs_default
Queue Name : Sub=sub-1:sla-1 3->3/1/1:1.1->6
FC Map : h2
Admin PIR : Max Oper PIR : Max
Admin MBS : 262500 B Oper MBS : 262656 B
HS Wrr Group : (not-applicable)
HS Wrr Class Weight: 1 HS Wrr Weight : 0
Depth : 233 KB
HS Class : 6 HS Alt Port Class Pool : No
HS Slope Policy : _tmnx_hs_default
-------------------------------------------------------------------------------
===============================================================================
A:PE-1#
Clear commands
Clear commands in the HSQ context are used to clear statistics associated with the HS secondary shaper:
clear port <port-id> hs-secondary-shaper <name> statistics
Resources monitoring
The following command is used to monitor resource on an HSQ IOM. Some of the resources (HSQ queue groups, HS primary shapers, HS secondary shapers, HS turbo queue groups, and so on) are allocated for internal use, thereby reducing the number of resources in the Free column. Such internally consumed resources are not available to be part of the user configuration.
The number of internally consumed resources depends on the configuration and MDA types.
In the following example, the number of allocated HS queue groups is 39. Each of the three subscribers consumes one HS queue group, which means that 36 HS queue groups are internally allocated. Similarly, out of 23 allocated HS secondary shapers, only 3 are user related (and visible via show commands): a "default" HS secondary shaper, HS secondary shaper 1, and HS secondary shaper 2. This means that 20 HS secondary shapers are internally consumed.
The same logic can be used for turbo HS queue groups and HS primary shapers. In addition, one managed HS primary shaper is always allocated per each HS secondary shaper. Both turbo HS queue groups and HS primary shapers are out of the scope of this chapter.
100G ports support port-based (access or network) HSQ queue groups on egress that can be configured to support higher throughput rates. Such port-based high rate HSQ queue groups on egress are referred to as egress HS turbo queue groups.
Hardware Resource Usage for Slot #3, CardType iom4-e-hs, Cmplx #0:
| Total | Allocated | Free
-------------------------------|-----------|-----------|------------
SAP Ingress QoS Policies | 1791| 1| 1790
Dynamic Egress Classification + 2047| 4| 2043
SAP Egress QoS Policies - | 4|
Network Egress Classification - | 0|
Ingress Queues | 131072| 501| 130571
Egress Queues | 786432| 137| 786295
Egress HS Turbo Queue Groups | 64| 10| 54
Egress HS Queue Groups | 98240| 39| 98201
Primary Shapers + 16384| 23| 16361
Explicit Primary Shapers - | 0|
Managed Primary Shapers - | 23|
Secondary Shapers | 4096| 23| 4073
Ingress Policers | 511999| 1| 511998
Egress Policers | 262143| 1| 262142
Ingress Policer Stats | 511967| 0| 511967
Egress Policer Stats | 262111| 0| 262111
Qos Ingress Root Arbiters | 65535| 1| 65534
Qos Egress Root Arbiters | 65535| 1| 65534
Qos Intermediate Arbiters | 262143| 0| 262143
Egress QoS Bypass | 131071| 0| 131071
Ingress ACL Entries | 65536| 2| 65534
Ingress QoS Entries | 16384| 2| 16382
Ingress IPv6 ACL Entries | 28672| 2| 28670
Ingress IPv6 QoS Entries | 4096| 2| 4094
Egress ACL Entries | 32768| 2| 32766
Egress QoS Entries | 14336| 2| 14334
Egress IPv6 ACL Entries | 16384| 2| 16382
Egress IPv6 QoS Entries | 2048| 2| 2046
Ingress ACL Filters | 2047| 0| 2047
Ingress IPv6 ACL Filters | 2047| 0| 2047
Egress ACL Filters | 2047| 0| 2047
Egress IPv6 ACL Filters | 2047| 0| 2047
QoS User Schedulers | 98303| 0| 98303
QoS User Scheduler Overrides | 196607| 0| 196607
Sap IngQGrp RedirLst Entries | 31999| 0| 31999
Dynamic Service Entries + 131071| 3| 131068
Subscriber Hosts - 131071| 3| 131068
Encap Group Members - 65535| 0| 65535
Egr Network Queue Group Mappings - 131071| 0| 131071
SapInst EgrQGrp RedirLst Entries - 31999| 0| 31999
Dynamic Nexthop Entries + 511999| 3| 511996
Subscriber Nexthops - 511999| 3| 511996
Ipsec tunnels - 511999| 0| 511999
Subscriber SPI QoS Overrides | 131072| 0| 131072
Mac Fdb Entries | 511999| 0| 511999
Egress TLS Mcast Entries | 368639| 1| 368638
Traffic management on HSQ
This section examines traffic output on HSQ IOM during congestion. Managing Congestion on HSQ in Single SLA Mode is a graphic representation of the configuration described previously, but with traffic streams running through the IOM.
Six traffic streams are sent in the downstream direction toward each of the three subscribers-in total there are 18 traffic streams. The traffic streams are shown on the left side of the figure, with their names, offered rates (IN column) and measured output rates (OUT column). The traffic streams are sent and analyzed by the traffic generator.
The three gray shaded squares in the center of the figure represent the three subscribers and their scheduling classes. The QoS hierarchy is shown on the right side. Red shaded areas (shapers) represent points of congestion on HSQ IOM caused by the 18 traffic streams.
Each of the six traffic streams per subscriber is fed into different subscriber queues. The first digit in the traffic stream name represents the subscriber, whereas the second digit represents the queue to which this stream is sent. For example, STRM 2-3 represents a traffic stream sent to sub-2, queue 3.
To summarize the scenario shown in Managing Congestion on HSQ in Single SLA Mode:
Traffic streams 1 and 2 of each subscriber are mapped to subscriber queues 1 and 2, which are in turn associated with WRR group 1 at the subscriber level.
WRR group 1 is, depending on the subscriber, rate limited to 20 Mb/s, 40 Mb/s, or 60 Mb/s. Weight ratio between queues 2 and 1 is 4:1. WRR group 1 is then attached to scheduling class 1.
Traffic stream 3 is via queue 3 directly mapped to scheduling class 3. Scheduling class 2 is unused in this example.
Traffic streams 4 and 5 are via queues 4 and 5 mapped to scheduling classes 4 and 5, which are at the port level collapsed into WRR group 1 with an aggregate rate limit of 60 Mb/s.
Traffic stream 6 is the highest priority stream that is via queue 6 mapped to scheduling class 6. At the port level, scheduling class 6 is rate limited to 20 Mb/s.
Subscriber (sub-1, sub-2, and sub-3) aggregate rates are set to 50 Mb/s, 75 Mb/s, and 100 Mb/s, respectively.
Subscribers sub-1 and sub-2 are mapped to HS secondary shaper 1 (via the outer VLANs on their SAPs) while sub-3 is mapped to HS secondary shaper 2.
HS port scheduler is rate limited to 200 Mb/s.
The configured rate limits at the subscriber level are L2 rates, whereas the configured rate limits at the HS secondary shaper level and the HS port level include L1 overhead and are, therefore, on-the-wire rates. On-the-wire rates account for 20 additional bytes in each Ethernet frame (8 bytes preamble and 12 bytes IFG).
All traffic streams are sent with constant rates (no added burstiness) and fixed packet size (1000 bytes). Therefore, the difference between the L2 rates and the on-the-wire rates for the 1000 byte packets is 1000/1020 = 0.98 or 2%. That is, on-the-wire rates are 2% higher than the L2 rates. The name for this 2% delta factor in this chapter will be the Rate Conversion Factor (RCF).
Input and output rates throughout the subscriber QoS hierarchy lists the input/output rates throughout the subscriber QoS hierarchy. Red shaded table cells represent congested objects. The numbers on the blue background represent the output rates measured on the traffic generator. The numbers in red (in parentheses) are configured aggregate rates for the object in the hierarchy. The numbers above them are operational rates measured by SR OS and observed via the show qos hs-scheduler-hierarchy command.
Some of the output columns in this table require additional explanation:
Per stream output rates are L2 rates as measured by the traffic generator. The resulting analysis will be based on these rates.
However, those rates can also be displayed via the show qos hs-scheduler-hierarchy subscriber <sub-name> egress command. RCF must be used to correctly interpret the results in respective rate domains (L1 versus L2).
WRR rates at the subscriber level (second column under the Output Rate section) are populated by manually adding rates measured by the traffic generator for traffic streams 1 and 2. The 4:1 indication in parentheses represents the weight ratio between the two streams. There are no means to observe the WRR group rates at the subscriber level directly within the system (via show commands).
Aggregate rates per subscriber (third column under the Output Rate section) are populated by manually adding the rates measured by the traffic generator from all six traffic streams for each subscriber. There are no means to observe the aggregate subscriber rate directly within the system (via show commands).
Aggregate rates per HS secondary shaper (fourth column under the Output Rate section) are on-the-wire rates displayed via the show qos hs-scheduler-hierarchy port 3/1/1 hs-secondary-shapers command. The displayed rates are rounded to the nearest Mb/s.
Per scheduling class rates at the port level (fifth column under the Output Rate section) are displayed via the show qos hs-scheduler-hierarchy port 3/1/1 command. The displayed rates are rounded by the system to the nearest Mb/s. The discrepancy between the actual rates for scheduling class 3 (60 Mb/s) and the displayed rate (61 Mb/s) is due to the measuring and rounding inaccuracy at display time. The actual rate at scheduling class 3 at the port level can be calculated by summing the measured rate of each traffic stream 3 for each subscriber and adjusting the sum for the on-the-wire rate (RCF).
The same logic applies to scheduling class 1 (actual rate 60 Mb/s versus 59 Mb/s measured rate).
The WRR 1 rate at the port level (sixth column under the Output Rate section) is manually calculated by adding the rates measured by the traffic generator of all streams mapped to scheduling classes 5 and 4 (6 traffic streams in total, two per each subscriber). At the port level, traffic on scheduling classes 5 and 4 is weighted in a 2:1 ratio (this is noted in parentheses).
The port rate (last column under the Output Rate section) is collected from two places:
The top number in obtained via the show qos hs-scheduler-hierarchy command and represents the on-the-wire rate.
The bottom number with blue background is the number measured by the traffic generator, which only measures L2 rates. On-the-wire rates can be converted to L2 rates by multiplying the on-the-wire rates by the RCF.
Subscr |
Strm |
Input rate in [Mb/s] |
Output rate in [Mb/s] |
|||||||
|---|---|---|---|---|---|---|---|---|---|---|
Per strm |
Agg per subscr |
Per strm (L2) |
WRR on subscr level |
Agg per subscr (L2) |
Agg per secondary shaper (wire) |
Per sch class on port level (wire) |
WRR-1 on port level (wire) |
Port |
||
Sub-1 |
1-6 |
10 |
110 |
6.54 |
- |
50 (50) |
120 (120) |
SC 6 20 (20) |
- |
200 (200) 196.08 (L2) |
1-5 |
15 |
13.07 |
- |
- |
||||||
1-4 |
15 |
6.54 |
- |
- |
||||||
1-3 |
20 |
20 |
- |
SC 5 40 |
60 (60) (2:1) |
|||||
1-2 |
25 |
3.1 |
3.9 (4:1) |
|||||||
1-1 |
25 |
0.8 |
||||||||
Sub-2 |
2-6 |
10 |
130 |
6.54 |
- |
67.65 (75) |
SC 4 20 |
|||
2-5 |
15 |
13.07 |
- |
|||||||
2-4 |
15 |
6.54 |
- |
|||||||
2-3 |
20 |
20 |
- |
SC 3 61 |
- |
|||||
2-2 |
35 |
17.2 |
21.5 (4:1) |
- |
||||||
2-1 |
35 |
4.3 |
- |
|||||||
Sub-3 |
3-6 |
10 |
160 |
6.54 |
- |
78.43 (100) |
80 (150) |
SC 2 0 |
- |
|
3-5 |
15 |
13.07 |
- |
- |
||||||
3-4 |
15 |
6.54 |
- |
- |
||||||
3-3 |
20 |
20 |
- |
SC 1 59 |
- |
|||||
3-2 |
50 |
25.8 |
32.3 (4:1) |
- |
||||||
3-1 |
50 |
6.5 |
- |
|||||||
Analysis of results
The analysis of results begins with the rates per stream measured by the traffic generator. The expected behavior is that those rates are in line with the theoretical rate calculations based on our understanding of QoS on HSQ IOM.
Considering the six strict priority classes in the HSQ scheduling mechanism, the expectation is that the traffic is serviced in the order of priority, from the highest scheduling class 6 to the lowest scheduling class 1. Consequently, the traffic analysis starts with the streams that are mapped to the highest priority scheduling class 6 (streams 1-6, 2-6, and 3-6), that is, stream 6 of sub-1, sub-2, and sub-3, respectively.
Scheduling class 6 (streams 1-6, 2-6, and 3-6) - the highest priority scheduling class
Due to the aggregate rate limit of 20 Mb/s for scheduling class 6 at the port level, it is expected that each subscriber receives an equal amount of traffic on scheduling class 6:
This slight discrepancy between the L2 and on-the-wire rates is common throughout the remaining analysis, and will not be repeated.
Scheduling classes 5 and 4 (streams 1-5, 1-4, 2-5, 2-4, 3-5, and 3-4)
Scheduling classes 5 and 4 are collapsed at the port level into WRR group 1, which is at the next scheduling priority to be served. These two scheduling classes are served by WRR group 1 in a 2:1 ratio. The aggregate rate limit for WRR 1 at the port level is set to 60 Mb/s.
The measured results in Input and output rates throughout the subscriber QoS hierarchy are 13 Mb/s on scheduling class 5 and 6.5 Mb/s on scheduling class 4 for traffic streams 4 and 5 of each individual subscriber. This is in line with the expected results (the slight difference is due to the measured L2 rates by the traffic generator and enforced on-the-wire rates at the port level).
Scheduling class 3 (streams 1-3, 2-3, and 3-3)
Traffic streams mapped to scheduling class 3 do not have any rate restriction at the scheduling class level. Those streams can be only limited by the congestion at the subscriber aggregate level, the HS secondary shaper aggregate level, or the port aggregate level. Because the total amount of traffic so far is below the congestion level at each point in the hierarchy, it is expected that traffic stream 3 flows unimpeded. Consequently, each subscriber should receive the full input rate of 20 Mb/s on scheduling class 3. The actual results in Input and output rates throughout the subscriber QoS hierarchy are aligned with the expected results.
To confirm that there is no congestion at the aggregate level so far for subscribers, HS secondary shapers, and the port, a calculation shows that each subscriber has received an equal amount of bandwidth so far: 46 Mb/s. This is below the configured aggregate rate limits at the subscriber, HS secondary scheduler, and port levels:
46 Mb/s is below the configured limit of 50 Mb/s for sub-1.
46 Mb/s is below the configured limit of 75 Mb/s for sub-2.
46 Mb/s is below the configured limit of 100 Mb/s for sub-3.
Sub-1 and sub-2 compete for the bandwidth at HS secondary shaper 1, and their combined rate of 92 Mb/s is below the configured aggregate rate of HS secondary shaper 1 (120 Mb/s).
Sub-3 with its 46 Mb/s is below its configured aggregate subscriber rate limit (100 Mb/s) and that of the HS secondary shaper 2 (150 Mb/s).
The total subscriber bandwidth of 3 x 46 Mb/s = 138 Mb/s is below the 200 Mb/s aggregate rate limit at the port level.
Scheduling classes 2 and 1 (streams 1-2, 1-1, 2-2, 2-1, 3-2, 3-1) - the lowest priority scheduling classes
The total amount of offered (input) traffic for lowest priority scheduling classes 1 and 2 across all three subscribers is 230 Mb/s (50 Mb/s for sub-1, 70 Mb/s for sub-2, and 100 Mb/s for sub-3). This amount of traffic will cause congestion at the aggregate level for sub-1, at the aggregate level of HS secondary shaper 1, and at the aggregate port level. The amount of traffic that each subscriber will receive for those two scheduling classes will now differ (up to this point, each subscriber received an equal amount of traffic).
Sub-1 will become limited by 50 Mb/s (the L2 rate limit) of its configured aggregate rate limit, which has only ~4 Mb/s left (with ~46 Mb/s received for sub-1). Considering that the scheduling classes 2 and 1 are serviced in 4:1 fashion, the ~4 Mb/s should be split between the two scheduling classes. Scheduling class 2 should receive ~3.2 Mb/s while scheduling class 1 should receive ~0.8 Mb/s. The measured data in Input and output rates throughout the subscriber QoS hierarchy shows that the expected and measured results on the traffic generator are aligned.
Up to this point, the spare capacity on the HS secondary shaper 1 is:
while the spare capacity at the sub-2 aggregate level is:
This means that the 70 Mb/s that sub-2 is sending on scheduling classes 2 and 1 will congest the HS secondary shaper 1 (~22 Mb/s left) before it congests the subscriber itself (~29 Mb/s left). The 22 Mb/s of on-the-wire bandwidth capacity left on the HS secondary shaper will be divided in 4:1 ratio between scheduling classes 2 and 1. Therefore, the scheduling class 2 should receive 17.6 Mb/s and the scheduling class 1 should receive 4.4 Mb/s. When converted to L2 rates, these expected results match the measured rates by the traffic generator in Input and output rates throughout the subscriber QoS hierarchy for sub-2.
At the subscriber level, rate enforcement is based on L2 packet size.
Spare capacity for HS secondary shaper 1 is calculated based on on-the-wire rates while the subscriber aggregate capacity is calculated in L2 rates, because the aggregate rate limit in the HS secondary shaper is configured in on-the-wire rates while the subscriber aggregate rates are configured in L2 rates.
Similar logic can be used for traffic on scheduling classes 2 and 1 for sub-3. The difference from the previous case is that this traffic will be limited by port congestion and not the HS secondary shaper congestion (or subscriber aggregate congestion).
The available bandwidth at the port level up to this point is:
The 100 Mb/s sent to sub-3 on scheduling classes 2 and 1 will be reduced to 33 Mb/s on the output in 4:1 ratio between scheduling classes 2 and 1. Scheduling class 2 should receive 26.3 Mb/s and scheduling class 1 should receive 6.6 Mb/s. Converted to L2 rates, these numbers match the measured rates by the traffic generator for streams 3-2 and 3-1 in Input and output rates throughout the subscriber QoS hierarchy .
Conclusion
This chapter described traffic management capabilities and configuration of HSQ IOM with ESM in single SLA mode. Because HSQ performs egress traffic management functions, the focus in this chapter was on egress QoS. The ingress QoS remains the same as on other non-HSQ IOMs where a combination of queues and policers can be deployed. Ingress and egress queues on HSQ are allocated from different queue pools that are separated at the hardware level (by different chips).
Some configuration practices are summarized here:
SAP queues in the ESM context should be removed in most subscriber/SAP (1:1) deployment scenarios (profiled-only-traffic command). This maximizes the number of ingress queues available for ESM, which is important because HSQ does not support shared queueing on ingress. The alternative to queueing is to use policers on ingress.
Queue depth (buffer size) can be statically configured (mbs command). By default, this parameter (MBS) is set dynamically and is somewhat high. To reduce the amount of buffering and delay, a smaller buffer size can be provisioned per queue.
The low-burst-max-class should be configured at the subscriber level and at the HS secondary scheduler level to ensure smaller buffering delays for higher priority traffic.
The delta between the high and low thresholds (hs-fixed-high-thresh-delta command) should be configured to accommodate a reasonable number of packets on higher priority classes that are serviced after the packets on lower priority classes have stopped due to crossing of the lower burst threshold. In this example, the value of 12 kbytes means that about 12 packets (each of 1000 byte size) on higher scheduling classes will be serviced unimpeded before the higher burst threshold is reached, which stops the service.
HSQ IOM provides scalable traffic management functions on egress. A high number of egress queues can be serviced in strict priority fashion and extensive shaping hierarchy provides protection against overrunning the link capacities at an access node level or at a subscriber level. HSQ IOM is a card for a scaled ESM (in single SLA mode) environment that requires a higher number of queues per subscriber with throughput demands in the range of 200 Gb/s per HSQ IOM.
Appendix A — Generic ESM configuration
Sub-profile, SLA-profile, and MSAP-policy configuration is omitted because it is already described in the Configuration section.
ESM configuration in this example starts with a subscriber-interface configured in an IES context. Subscriber hosts are instantiated in IES 3 service, under the group-interface "group-int-1", which is created under the subscriber-interface "sub-int-1". Authentication and address assignment of the subscriber hosts is performed via LUDB user-db "user-db-1". The IP addresses that are assigned to the hosts are statically configured in LUDB (DHCP server is not used in this setup).
configure
service
ies 3 name "3" customer 1 create
subscriber-interface "sub-int-1" create
address 10.10.1.254/24
group-interface "group-int-1" create
dhcp
proxy-server
emulated-server 10.10.1.254
no shutdown
exit
option
action keep
circuit-id
remote-id
exit
trusted
lease-populate 100
gi-address 10.10.1.254
user-db "user-db-1"
no shutdown
exit
exit
no shutdown
exit
exit
Subscriber SAPs are automatically created based on the VLAN tags carried in the initial control packets of the subscriber hosts. This VLAN auto-detection and SAP auto-creation is configured under the capture SAP hierarchy. The capture SAP is configured to support LUDB authentication for dynamic DHCPv4 host instantiation:
configure
service
vpls 10 name "10" customer 1 create
sap 3/1/1:*.* capture-sap create
trigger-packet dhcp
dhcp-user-db "user-db-1"
exit
exit
Sub-ident-policy is a mandatory configuration in ESM. It determines the mapping method between the sub/SLA profiles and the corresponding strings obtained during the authentication phase for the subscriber. Subscribers strings obtained during the authentication phase point, in some form (determined by sub-ident-policy), to the configured sub/SLA profiles (in the SR OS node) that will be associated with the subscriber.
configure
subscriber-mgmt
sub-ident-policy "sub_ident_pol" create
sub-profile-map
use-direct-map-as-default
exit
sla-profile-map
use-direct-map-as-default
exit
exit
In this example, authentication of the subscriber hosts and IP address assignment is performed through LUDB. The hosts are identified based on the circuit-id and remote-id fields in DHCP control packets. Sub/SLA -profile strings in LUDB are directly mapped to the configured sub/SLA-profiles in the SR OS node. This direct mapping is implied by the use-direct-map-as-default command within the sub-ident-policy.
Service ID, group-interface name, and MSAP policy name for the subscriber is also determined during the authentication phase via LUDB. LUDB carries only ESM-specific configuration. There is no HSQ relevant configuration present in LUDB.
configure
subscriber-mgmt
local-user-db "user-db-1" create
ipoe
match-list circuit-id remote-id
host "sub-1-host-1" create
host-identification
circuit-id string "sub-1"
remote-id string "host-1"
exit
address 10.10.1.1
identification-strings 254 create
subscriber-id "sub-1"
sla-profile-string "sla-1"
sub-profile-string "sub-prof-1"
exit
msap-defaults
group-interface "group-int-1"
policy "msaps"
service 3
exit
options
subnet-mask 255.255.255.0
exit
no shutdown
exit
host "sub-2-host-1" create
host-identification
circuit-id string "sub-2"
remote-id string "host-1"
exit
address 10.10.1.2
identification-strings 254 create
subscriber-id "sub-2"
sla-profile-string "sla-2"
sub-profile-string "sub-prof-2"
exit
msap-defaults
group-interface "group-int-1"
policy "msaps"
service 3
exit
options
subnet-mask 255.255.255.0
exit
no shutdown
exit
host "sub-3-host-1" create
host-identification
circuit-id string "sub-3"
remote-id string "host-1"
exit
address 10.10.1.3
identification-strings 254 create
subscriber-id "sub-3"
sla-profile-string "sla-3"
sub-profile-string "sub-prof-3"
exit
msap-defaults
group-interface "group-int-1"
policy "msaps"
service 3
exit
options
subnet-mask 255.255.255.0
exit
no shutdown
exit
exit
no shutdown
exit
exit