Distributed CPU Protection

This chapter describes Distributed CPU Protection (DCP) configurations.

Topics in this chapter include:

Applicability

This chapter was originally written for SR OS Release 11.0R1. The CLI in the current edition corresponds to Release 15.0.R1.

Overview

SR OS provides several rate limiting mechanisms to protect the CPM/CFM processing resources of the router:

  • CPU Protection: A centralized rate limiting function that operates on the CPM to limit traffic destined to the CPUs.

  • Distributed CPU Protection: A control traffic rate limiting protection mechanism for the CPM/CFM that operates on the line cards (hence ‛distributed’). CPU protection protects the CPU of the node that it is configured on from a DOS attack by limiting the amount of traffic coming in from one of its ports and destined to the CPM (to be processed by its CPU) using a combination of the configurable limits.

The goal of this chapter is to familiarize the reader with the configuration and use of DCP. A simple and controlled setup is used to illustrate how the protection behaves and how to use the tools provided for the feature.

External testing equipment (‟tester”) is used to send control traffic of various protocols at various rates to the router in order to exercise DCP. Log events and show routines are examined to explain the indications that the router provides to an operator.

Configuration

The test topology is shown in Test Topology. A 10Gb Ethernet link is used between the tester and the router.

Figure 1. Test Topology
  1. The basic configuration of the MDA, port, interface and a security event log on the router is as follows.

    configure
        card 4
            card-type iom3-xp-b
            mda 1
                mda-type m4-10gb-xp-xfp
                no shutdown
            exit
        exit
    exit
    
    configure
        port 4/1/1
            ethernet
            exit
            no shutdown
        exit
    exit
    
    configure
        router Base
            interface "int-R1-T1.4.4"
                address 192.168.40.1/24
                description "to tester T1, port 4.4"
                port 4/1/1
                no shutdown
            exit
        exit
    exit
    
    configure
        log 
            log-id 15
                from security 
                to memory 1024
                no shutdown
            exit
        exit 
    exit
    

    This chapter was originally developed on a 7750 SR-c12 platform but it is equally applicable to other platforms such as the 7750 SR-7/12. If other platforms, such as the 7750 SR-7/12 that support centralized CPU Protection, are used to explore DCP then the centralized CPU Protection should be disabled (for the purposes of this chapter) so that it does not interfere with reproducing the same results as described below. In a normal production network, CPU Protection and DCP are complimentary and can be used together. To disable centralized CPU Protection for the purposes of reproducing the results below please ensure that:

    • protocol-protection is disabled.

    • All rates in all polices (including any default polices) are configured to max.

  2. In order to activate DCP a policy is created and assigned to the interface.

    The first policy that is used in this chapter is used to simply count protocol packets to see that they are indeed flowing from the tester to the router and being extracted and identified.

    The dcp-policy-count policy is configured as follows:

    configure
        system
            security
                dist-cpu-protection
                    policy "dcp-policy-count" create
                        description "Static policers with rate 0 for counting packets"
                        static-policer "sp-arp" create
                            description "static policer for ARP"
                            rate packets 0 within 1
                        exit
                        static-policer "sp-icmp" create
                            description "static policer for ICMP"
                            rate packets 0 within 1
                        exit
                        static-policer "sp-igmp" create
                            description "static policer for IGMP"
                            rate packets 0 within 1
                        exit
                        protocol arp create
                            enforcement static "sp-arp"
                        exit
                        protocol icmp create
                            enforcement static "sp-icmp"
                        exit
                        protocol igmp create
                            enforcement static "sp-igmp"
                        exit
                    exit
                exit
            exit
        exit
    exit
    

    For the dcp-policy-count policy configuration:

    • The policy contains three static policers: sp-arp, sp-icmp and sp-igmp. These policers are then used by the three configured protocols that are part of the policy: arp, icmp and igmp.

    • The list of protocols that are applicable to DCP are as follows: arp, dhcp, http-redirect, icmp, igmp, mld, ndis, pppoe-pppoa, all-unspecified, mpls-ttl, bfd-cpm, bgp, eth-cfm, isis, ldp, ospf, pim and rsvp. The all-unspecified protocol is a special ‟catch-all”. See the 7450 ESS, 7750 SR, 7950 XRS, and VSR System Management Guide for more details.

    • This policy instantiates three permanent (static) policers for every object (for example, interface) that the policy is associated with.

    • The three protocols each reference their own static policer so each protocol will be independently rate limited. A single static policer can also be used to rate limit multiple protocols but that capability is not used in this chapter.

    • The rate is set to 0 which means all packets will be considered as non-conformant to the policer. This configuration is used to provide counters of protocol packets. The DCP counters provide the count of packets exceeding the policing parameters since the given policer was previously declared as conformant or newly instantiated. A rate of zero ensures that the policer will never be declared as conformant and hence will never reset the counters.

    • The exceed-action is not configured and takes the default value of none. The log-events parameter is not configured and is enabled by default. That means the policer will notify the operator when the first packet arrives but will not discard or mark any packets.

  3. Assign the dcp-policy-count to the interface:

    *A:R1# configure router interface "int-R1-T1.4.4"
    *A:R1>config>router>if# dist-cpu-protection "dcp-policy-count"
    
  4. Examine some log and status on the router to get a baseline (no traffic is flowing from the tester to the router at this point). Notice that the CPU utilization is fairly low with an overall Idle of 94% and no task groups at more than 2.5% capacity usage. Future example output from this show routine will be snipped to only show relevant and interesting lines.

    *A:R1# show system cpu 
    
    ===============================================================================
    CPU Utilization (Sample period: 1 second)
    ===============================================================================
    Name                                   CPU Time       CPU Usage        Capacity
                                             (uSec)                           Usage
    -------------------------------------------------------------------------------
    BFD                                          60          ~0.00%          ~0.00%
    BGP                                      30,892           0.34%           0.62%
    BGP PE-CE                                     0           0.00%           0.00%
    CALLTRACE                                 5,210           0.05%           0.51%
    CFLOWD                                    5,128           0.05%           0.51%
    Cards & Ports                            39,591           0.44%           0.95%
    DHCP Server                                  35          ~0.00%          ~0.00%
    ETH-CFM                                   4,584           0.05%           0.46%
    HQoS Algorithm                                0           0.00%           0.00%
    HQoS Statistics                               0           0.00%           0.00%
    ICC                                       1,225           0.01%           0.12%
    IGMP/MLD                                  1,080           0.01%           0.10%
    IMSI Db Appl                                258          ~0.00%           0.01%
    IOM                                           0           0.00%           0.00%
    IP Stack                                 56,965           0.63%           0.51%
    IS-IS                                    51,342           0.57%           0.60%
    ISA                                      16,173           0.18%           0.55%
    LDP                                      31,118           0.34%           0.55%
    Logging                                      53          ~0.00%          ~0.00%
    MBUF                                          0           0.00%           0.00%
    MCS                                         536          ~0.00%           0.04%
    MPLS/RSVP                                 8,915           0.09%           0.57%
    MSCP                                          0           0.00%           0.00%
    MSDP                                          0           0.00%           0.00%
    Management                               18,039           0.20%           0.73%
    OAM                                      12,422           0.13%           0.48%
    OSPF                                    118,279           1.32%           0.58%
    OpenFlow                                  1,037           0.01%           0.01%
    PIM/L2Mcast                                   0           0.00%           0.00%
    PKI                                         272          ~0.00%           0.02%
    PTP                                          71          ~0.00%          ~0.00%
    RIP                                           0           0.00%           0.00%
    RTM/Policies                                  0           0.00%           0.00%
    Redundancy                               10,321           0.11%           0.63%
    SNMP Daemon                                   0           0.00%           0.00%
    Security                                      0           0.00%           0.00%
    Services                                  8,775           0.09%           0.52%
    Stats                                         0           0.00%           0.00%
    Subscriber Mgmt                           6,439           0.07%           0.14%
    System                                   94,262           1.05%           2.28%
    Traffic Eng                                   0           0.00%           0.00%
    VRRP                                      1,942           0.02%           0.12%
    WEB Redirect                                 95          ~0.00%          ~0.00%
    -------------------------------------------------------------------------------
    Total                                 8,936,439         100.00%                
       Idle                               8,411,320          94.12%                
       Usage                                525,119           5.87%                
    Busiest Core Utilization                 79,550           8.01%                
    ===============================================================================
    *A:R1# 
    

    The DCP feature is reporting no violations for interfaces on card 4.

    *A:R1# tools dump security dist-cpu-protection violators enforcement interface card 4 
    ===============================================================================
    Distributed Cpu Protection Current Interface Enforcer Policer Violators
    ===============================================================================
    Interface                       Policer/Protocol                    Hld Rem       
    -------------------------------------------------------------------------------
    -------------------------------------------------------------------------------
    Violators on Slot-4 Fp-1
    -------------------------------------------------------------------------------
    -------------------------------------------------------------------------------
    [S]-Static [D]-Dynamic [M]-Monitor
    -------------------------------------------------------------------------------
    ===============================================================================
    *A:R1#
    

    There are no security log events.

    *A:R1# show log log-id 15 
    ===============================================================================
    Event Log 15
    ===============================================================================
    Description : (Not Specified)
    Memory Log contents  [size=1024   next event=1  (not wrapped)]
    *A:R1#
    

    The detailed DCP status for the interface shows all three policers are currently in the conform state.

    *A:R1# show router interface "int-R1-T1.4.4" dist-cpu-protection 
    
    ===============================================================================
    Interface "int-R1-T1.4.4" (Router: Base)
    ===============================================================================
    Distributed CPU Protection Policy :  dcp-policy-count
    
    -------------------------------------------------------------------------------
    Statistics/Policer-State Information
    ===============================================================================
    -------------------------------------------------------------------------------
    Static Policer
    -------------------------------------------------------------------------------
    Policer-Name        : sp-arp 
    Card/FP             : 4/1               Policer-State       : Conform
    Protocols Mapped    : arp  
    Exceed-Count        : 0                 
    Detec. Time Remain  : 0 seconds         Hold-Down Remain.   : none
    
    Policer-Name        : sp-icmp 
    Card/FP             : 4/1               Policer-State       : Conform
    Protocols Mapped    : icmp  
    Exceed-Count        : 0                 
    Detec. Time Remain  : 0 seconds         Hold-Down Remain.   : none
    
    Policer-Name        : sp-igmp 
    Card/FP             : 4/1               Policer-State       : Conform
    Protocols Mapped    : igmp  
    Exceed-Count        : 0                 
    Detec. Time Remain  : 0 seconds         Hold-Down Remain.   : none
    -------------------------------------------------------------------------------
    -------------------------------------------------------------------------------
    Local-Monitoring Policer
    -------------------------------------------------------------------------------
    No entries found
    -------------------------------------------------------------------------------
    -------------------------------------------------------------------------------
    Dynamic-Policer (Protocol)
    -------------------------------------------------------------------------------
    No entries found
    -------------------------------------------------------------------------------
    ===============================================================================
    *A:R1#
    
  5. Configure the tester to send ARP, ICMP and IGMP traffic to the router using the following rates:

    • ARP: 2 packets per second (pps)

    • ICMP: 4 pps

    • IGMP: 8 pps

    Here are some tips for how to configure the tester to send protocol packets that will be recognized by the router:

    • ARP:

      • Set the MAC destination address to FF-FF-FF-FF-FF-FF

      • Use an ARP Request format

    • ICMP:

      • Use an ICMP type of 8 (echo request, such as ping).

      • Set the MAC destination address equal to the MAC address of the receiving port. The MAC address of port 4/1/1 can be seen in the output of show port 4/1/1 as the configured address.

      • Set the IP destination address to 192.168.40.1 and the IP source address to 192.168.40.2.

    • IGMP:

      • Set the MAC destination address equal to the MAC address of the receiving port. The MAC address of port 4/1/1 can be seen in the output of show port 4/1/1 as the configured address.

      • Set the IP destination address to 224.0.0.2 and the IP source address to 0.0.0.0.

      • Set the IGMP version to 2, make the IGMP message type a Membership Query to Group 0.

    Also ensure that the tester interleaves the three streams of protocol packets such that it schedules them independently in an interleaved fashion, not serially.

    Figure 2. Count Traffic with DCP Policy Count
  6. Notice that DCP now reports some violations of the policy against the interface.

    *A:R1# tools dump security dist-cpu-protection violators enforcement interface card 4 
    ===============================================================================
    Distributed Cpu Protection Current Interface Enforcer Policer Violators
    ===============================================================================
    Interface                       Policer/Protocol                    Hld Rem       
    -------------------------------------------------------------------------------
    -------------------------------------------------------------------------------
    Violators on Slot-4 Fp-1
    -------------------------------------------------------------------------------
    int-R1-T1.4.4                   sp-arp                          [S] none         
    int-R1-T1.4.4                   sp-icmp                         [S] none         
    int-R1-T1.4.4                   sp-igmp                         [S] none         
    -------------------------------------------------------------------------------
    [S]-Static [D]-Dynamic [M]-Monitor
    -------------------------------------------------------------------------------
    ===============================================================================
    *A:R1# 
    

    After a few seconds, the DCP exceed-count values can be seen incrementing.

    Note the following details:

    • Exceed-Count is non-zero. This will continue incrementing and will never reset since the rate configured in the DCP policy is zero.

    • The Policer-State is Exceed. The policers have detected that the protocol is non-conformant to the configured rate.

    • Detec. Time Remain stays at 29 seconds. This countdown timer is automatically reset to 30 seconds every time a policer is detected as non-conformant (which will be continually when the rate is set to 0 and packets of that protocol are being received).

    *A:R1# show router interface "int-R1-T1.4.4" dist-cpu-protection 
    
    ===============================================================================
    Interface "int-R1-T1.4.4" (Router: Base)
    ===============================================================================
    Distributed CPU Protection Policy :  dcp-policy-count
    
    -------------------------------------------------------------------------------
    Statistics/Policer-State Information
    ===============================================================================
    -------------------------------------------------------------------------------
    Static Policer
    -------------------------------------------------------------------------------
    Policer-Name        : sp-arp 
    Card/FP             : 4/1               Policer-State       : Exceed
    Protocols Mapped    : arp  
    Exceed-Count        : 263               
    Detec. Time Remain  : 29 seconds        Hold-Down Remain.   : none
    
    Policer-Name        : sp-icmp 
    Card/FP             : 4/1               Policer-State       : Exceed
    Protocols Mapped    : icmp  
    Exceed-Count        : 525               
    Detec. Time Remain  : 29 seconds        Hold-Down Remain.   : none
    
    Policer-Name        : sp-igmp 
    Card/FP             : 4/1               Policer-State       : Exceed
    Protocols Mapped    : igmp  
    Exceed-Count        : 1050              
    Detec. Time Remain  : 29 seconds        Hold-Down Remain.   : none
    -------------------------------------------------------------------------------
    -------------------------------------------------------------------------------
    Local-Monitoring Policer
    -------------------------------------------------------------------------------
    No entries found
    -------------------------------------------------------------------------------
    -------------------------------------------------------------------------------
    Dynamic-Policer (Protocol)
    -------------------------------------------------------------------------------
    No entries found
    -------------------------------------------------------------------------------
    ===============================================================================
    *A:R1# 
    
  7. Keep the tester running.

    Now a DCP policy that enforces protocol rates using static policers will be applied to the interface. First, the policy is created:

    configure
        system
            security
                dist-cpu-protection
                    policy "dcp-static-policy-1" create
                        description "Static policers for arp, icmp and igmp"
                        static-policer "sp-arp" create
                            rate packets 10 within 1
                            exceed-action discard
                        exit
                        static-policer "sp-icmp" create
                            rate packets 20 within 1
                            exceed-action discard
                        exit
                        static-policer "sp-igmp" create
                            rate packets 10 within 1
                            exceed-action discard
                        exit
                        protocol arp create
                            enforcement static "sp-arp"
                        exit
                        protocol icmp create
                            enforcement static "sp-icmp"
                        exit
                        protocol igmp create
                            enforcement static "sp-igmp"
                        exit
                    exit
                exit
            exit
        exit
    exit
    

    For the dcp-static-policy-1 policy configuration, note that a few parameters are different than in the previously created dcp-policy-count policy:

    • The rates are set to low (but non-zero) values.

    • The exceed-action is configured as discard such that packets are dropped once the rate is exceeded.

    Now assign the policy to the test interface:

    *A:R1# configure router interface "int-R1-T1.4.4" 
                            dist-cpu-protection "dcp-static-policy-1"
    
    *A:R1# show system security dist-cpu-protection policy "dcp-static-policy-1" association 
    
    ===============================================================================
    Distributed CPU Protection Policy
    ===============================================================================
    Policy Name : dcp-static-policy-1
    Description : Static policers for arp, icmp and igmp
    
    -------------------------------------------------------------------------------
    Associations
    -------------------------------------------------------------------------------
    
    SAP associations
    -------------------------------------------------------------------------------
      None
    
    Managed SAP associations
    -------------------------------------------------------------------------------
      None
    
    Interface associations
    -------------------------------------------------------------------------------
    Router-Name : Base                      
      int-R1-T1.4.4
    -------------------------------------------------------------------------------
    Number of interfaces : 1
    ===============================================================================
    *A:R1#
    
  8. Increase the rate of IGMP packets that the tester is sending to 1000pps (keep ARP and ICMP at 2pps and 4pps).

    Figure 3. Limit Traffic with dcp-static-policy-1
  9. Notice that the system has identified a violation of the DCP rates for the IGMP policer.

    *A:R1# tools dump security dist-cpu-protection violators enforcement interface card 4 
    ===============================================================================
    Distributed Cpu Protection Current Interface Enforcer Policer Violators
    ===============================================================================
    Interface                       Policer/Protocol                    Hld Rem       
    -------------------------------------------------------------------------------
    -------------------------------------------------------------------------------
    Violators on Slot-4 Fp-1
    -------------------------------------------------------------------------------
    int-R1-T1.4.4                   sp-igmp                         [S] none         
    -------------------------------------------------------------------------------
    [S]-Static [D]-Dynamic [M]-Monitor
    -------------------------------------------------------------------------------
    ===============================================================================
    *A:R1#
    

    After a few minutes, the violation will be indicated as a log event. This delay is due to the design of DCP. In order to support large scale operation of DCP, and also to avoid overload conditions, a polling process is used to monitor state changes in the policers and to gather violations. This means there can be a delay between when an event occurs in the data plane and when the relevant state change or event notification occurs towards an operator, but in the meantime the policers are still operating and protecting the control plane.

    *A:R1# show log log-id 15 
    ===============================================================================
    Event Log 15
    ===============================================================================
    Description : (Not Specified)
    Memory Log contents  [size=1024   next event=2  (not wrapped)]
    
    1 2017/04/27 09:47:53.21 CEST WARNING: SECURITY #2066 Base DCPUPROT
    "Non conformant network_if "int-R1-T1.4.4" on fp 4/1 detected at 04/27/2017 09:47:07.
     Policy "dcp-static-policy-1". Policer="sp-igmp"(static). Excd count=411"
    *A:R1# 
    
    *A:R1# show router interface "int-R1-T1.4.4" dist-cpu-protection 
    
    ===============================================================================
    Interface "int-R1-T1.4.4" (Router: Base)
    ===============================================================================
    Distributed CPU Protection Policy :  dcp-static-policy-1
    
    -------------------------------------------------------------------------------
    Statistics/Policer-State Information
    ===============================================================================
    -------------------------------------------------------------------------------
    Static Policer
    -------------------------------------------------------------------------------
    Policer-Name        : sp-arp 
    Card/FP             : 4/1               Policer-State       : Conform
    Protocols Mapped    : arp  
    Exceed-Count        : 0                 
    Detec. Time Remain  : 0 seconds         Hold-Down Remain.   : none
    
    Policer-Name        : sp-icmp 
    Card/FP             : 4/1               Policer-State       : Conform
    Protocols Mapped    : icmp  
    Exceed-Count        : 0                 
    Detec. Time Remain  : 0 seconds         Hold-Down Remain.   : none
    
    Policer-Name        : sp-igmp 
    Card/FP             : 4/1               Policer-State       : Exceed
    Protocols Mapped    : igmp  
    Exceed-Count        : 640151            
    Detec. Time Remain  : 29 seconds        Hold-Down Remain.   : none
    -------------------------------------------------------------------------------
    -------------------------------------------------------------------------------
    Local-Monitoring Policer
    -------------------------------------------------------------------------------
    No entries found
    -------------------------------------------------------------------------------
    -------------------------------------------------------------------------------
    Dynamic-Policer (Protocol)
    -------------------------------------------------------------------------------
    No entries found
    -------------------------------------------------------------------------------
    ===============================================================================
    *A:R1#
    

    The status of DCP on the interface also shows the igmp policer as being in an Exceed state:

    The CPU utilization of the IGMP task group is not impacted because DCP is discarding packets that are non-conformant to the configure rate.

    *A:R1# show system cpu 
    
    ===============================================================================
    CPU Utilization (Sample period: 1 second)
    ===============================================================================
    Name                                   CPU Time       CPU Usage        Capacity
                                             (uSec)                           Usage
    -------------------------------------------------------------------------------
    BFD                                         160          ~0.00%           0.01%
    ---snip---
    IGMP/MLD                                  1,795           0.02%           0.09%
    ---snip---
    Stats                                         0           0.00%           0.00%
    Subscriber Mgmt                           5,661           0.06%           0.09%
    System                                   77,189           0.86%           1.23%
    Traffic Eng                                   0           0.00%           0.00%
    VRRP                                      1,679           0.01%           0.09%
    WEB Redirect                                 83          ~0.00%          ~0.00%
    -------------------------------------------------------------------------------
    Total                                 8,936,280         100.00%                
       Idle                               8,420,519          94.22%                
       Usage                                515,761           5.77%                
    Busiest Core Utilization                 78,908           7.94%                
    ===============================================================================
    *A:R1# 
    
  10. Remove the DCP policy from the interface and see the capacity usage going up for the IGMP task group.

    *A:R1# configure router interface "int-R1-T1.4.4" no dist-cpu-protection 
    *A:R1# 
    
    *A:R1# show system cpu 
    
    ===============================================================================
    CPU Utilization (Sample period: 1 second)
    ===============================================================================
    Name                                   CPU Time       CPU Usage        Capacity
                                             (uSec)                           Usage
    -------------------------------------------------------------------------------
    BFD                                         191          ~0.00%           0.01%
    ---snip---
    ICC                                       1,332           0.01%           0.13%
    IGMP/MLD                                 78,184           0.87%           7.78%
    IMSI Db Appl                                249          ~0.00%          ~0.00%
    IOM                                           0           0.00%           0.00%
    IP Stack                                183,448           2.05%           9.16%
    IS-IS                                    49,866           0.55%           0.56%
    ---snip---
    Subscriber Mgmt                           8,153           0.09%           0.14%
    System                                  144,738           1.62%           6.11%
    Traffic Eng                                   0           0.00%           0.00%
    VRRP                                      2,396           0.02%           0.15%
    WEB Redirect                                 83          ~0.00%          ~0.00%
    -------------------------------------------------------------------------------
    Total                                 8,925,786         100.00%                
       Idle                               8,123,698          91.01%                
       Usage                                802,088           8.98%                
    Busiest Core Utilization                170,131          17.15%                
    ===============================================================================
    *A:R1# 
    
  11. Increase the rate of IGMP traffic from the tester to 5000 pps. See the CPU utilization increase further.

    *A:R1# show system cpu 
    
    ===============================================================================
    CPU Utilization (Sample period: 1 second)
    ===============================================================================
    Name                                   CPU Time       CPU Usage        Capacity
                                             (uSec)                           Usage
    -------------------------------------------------------------------------------
    BFD                                         158          ~0.00%           0.01%
    ---snip---
    ICC                                       1,061           0.01%           0.10%
    IGMP/MLD                                398,106           4.44%          39.99%
    IMSI Db Appl                                142          ~0.00%          ~0.00%
    IOM                                           0           0.00%           0.00%
    IP Stack                                648,378           7.24%          43.68%
    IS-IS                                    59,623           0.66%           0.65%
    ---snip---
    Subscriber Mgmt                           7,308           0.08%           0.13%
    System                                  364,124           4.06%          28.73%
    Traffic Eng                                   0           0.00%           0.00%
    VRRP                                      2,156           0.02%           0.12%
    WEB Redirect                                117          ~0.00%           0.01%
    -------------------------------------------------------------------------------
    Total                                 8,951,453         100.00%                
       Idle                               7,114,732          79.48%                
       Usage                              1,836,721          20.51%                
    Busiest Core Utilization                590,342          59.35%                
    ===============================================================================
    *A:R1# 
    
  12. Reinstall the DCP policy to the interface and see the CPU utilization drop.

    *A:R1# configure router interface "int-R1-T1.4.4"                         dist-cpu-protection "dcp-static-policy-1"
    
    *A:R1# show system cpu 
    
    ===============================================================================
    CPU Utilization (Sample period: 1 second)
    ===============================================================================
    Name                                   CPU Time       CPU Usage        Capacity
                                             (uSec)                           Usage
    -------------------------------------------------------------------------------
    BFD                                          72          ~0.00%          ~0.00%
    ---snip---
    ICC                                       1,000           0.01%           0.10%
    IGMP/MLD                                  2,149           0.02%           0.12%
    IMSI Db Appl                                166          ~0.00%          ~0.00%
    IOM                                           0           0.00%           0.00%
    IP Stack                                 60,407           0.67%           0.55%
    IS-IS                                    50,966           0.57%           0.58%
    ---snip---
    Subscriber Mgmt                           5,847           0.06%           0.09%
    System                                   96,233           1.07%           2.23%
    Traffic Eng                                   0           0.00%           0.00%
    VRRP                                      2,338           0.02%           0.15%
    WEB Redirect                                 94          ~0.00%          ~0.00%
    -------------------------------------------------------------------------------
    Total                                 8,925,256         100.00%                
       Idle                               8,396,016          94.07%                
       Usage                                529,240           5.92%                
    Busiest Core Utilization                 81,620           8.22%                
    ===============================================================================
    *A:R1# 
    
  13. Stop the tester from sending packets, wait a few minutes and then note the status of the system.

    There are no longer any violations of any enforcement policers on any interfaces on card 1.

    *A:R1# tools dump security dist-cpu-protection violators enforcement interface card 4 
    ===============================================================================
    Distributed Cpu Protection Current Interface Enforcer Policer Violators
    ===============================================================================
    Interface                       Policer/Protocol                    Hld Rem       
    -------------------------------------------------------------------------------
    -------------------------------------------------------------------------------
    Violators on Slot-4 Fp-1
    -------------------------------------------------------------------------------
    -------------------------------------------------------------------------------
    [S]-Static [D]-Dynamic [M]-Monitor
    -------------------------------------------------------------------------------
    ===============================================================================
    *A:R1# 
    

    The IGMP policer is indicated as conformant in the log events.

    *A:R1# show log log-id 15 
    
    ===============================================================================
    Event Log 15
    ===============================================================================
    Description : (Not Specified)
    Memory Log contents  [size=1024   next event=4  (not wrapped)]
    
    3 2017/04/27 10:02:53.25 CEST WARNING: SECURITY #2072 Base DCPUPROT
    "Network_if "int-R1-T1.4.4" on fp 4/1 newly conformant at 04/27/2017 10:02:04. Policy
     "dcp-static-policy-1". Policer="sp-igmp"(static). Excd count=227756"
    
    ---snip---
    
    *A:R1#
    

    The interface DCP details show all policers as conformant.

    *A:R1# show router interface "int-R1-T1.4.4" dist-cpu-protection 
    
    ===============================================================================
    Interface "int-R1-T1.4.4" (Router: Base)
    ===============================================================================
    Distributed CPU Protection Policy :  dcp-static-policy-1
    
    -------------------------------------------------------------------------------
    Statistics/Policer-State Information
    ===============================================================================
    -------------------------------------------------------------------------------
    Static Policer
    -------------------------------------------------------------------------------
    Policer-Name        : sp-arp 
    Card/FP             : 4/1               Policer-State       : Conform
    Protocols Mapped    : arp  
    Exceed-Count        : 0                 
    Detec. Time Remain  : 0 seconds         Hold-Down Remain.   : none
    
    Policer-Name        : sp-icmp 
    Card/FP             : 4/1               Policer-State       : Conform
    Protocols Mapped    : icmp  
    Exceed-Count        : 0                 
    Detec. Time Remain  : 0 seconds         Hold-Down Remain.   : none
    
    Policer-Name        : sp-igmp 
    Card/FP             : 4/1               Policer-State       : Conform
    Protocols Mapped    : igmp  
    Exceed-Count        : 0                 
    Detec. Time Remain  : 0 seconds         Hold-Down Remain.   : none
    -------------------------------------------------------------------------------
    -------------------------------------------------------------------------------
    Local-Monitoring Policer
    -------------------------------------------------------------------------------
    No entries found
    -------------------------------------------------------------------------------
    -------------------------------------------------------------------------------
    Dynamic-Policer (Protocol)
    -------------------------------------------------------------------------------
    No entries found
    -------------------------------------------------------------------------------
    ===============================================================================
    *A:R1# 
    

    An optional hold-down can be used in the configuration of the exceed-action of the policers in order to apply the exceed-action for a defined period (even if the policer goes conformant again during that period). The hold-down could be used, for, to discard all packets associated with a policer for one hour after a violation is detected. An ‟indefinite” period is also supported which enforces discard or marking until the operator clears the policer with the tools perform security dist-cpu-protection release-hold-down command.

  14. The next scenario explored in this chapter is the use of DCP dynamic enforcement.

    In order to use dynamic enforcement policers, a number of dynamic policers must be allocated to the DCP pool for the particular card being used.

    *A:R1# configure card 4 fp dist-cpu-protection dynamic-enforcement-policer-pool 1000 
    *A:R1# 
    

    The number allocated should be greater than the maximum number of dynamic policers expected to be in use on the card at one time. A conservative (large) number could be selected at first, and then the following show command can give data to help tune the pool to a smaller size over time:

    *A:R1# show card 4 fp 1 dist-cpu-protection 
    
    ===============================================================================
    Card : 4 Forwarding Plane(FP) : 1
    ===============================================================================
    Dynamic Enforcement Policer Pool : 1000
    -------------------------------------------------------------------------------
    
    -------------------------------------------------------------------------------
    Statistics Information
    -------------------------------------------------------------------------------
    Dynamic-Policers Currently In Use      : 0
    Hi-WaterMark Hit Count                 : 0
    Hi-WaterMark Hit Time                  : 04/27/2017 10:08:24 UTC
    Dynamic-Policers Allocation Fail Count : 0
    -------------------------------------------------------------------------------
    ===============================================================================
    *A:R1#
    

    If the dynamic-enforcement-policer-pool is too small then when a local-monitoring-policer detects violating traffic, the dynamic enforcement policers will not be able to be instantiated. A log event will warn the operator when the pool is nearly exhausted.

    A sample dynamic enforcement policy is created as follows:

    configure
        system
            security
                dist-cpu-protection
                    policy "dcp-dynamic-policy-1" create
                        description "Dynamic policing policy"
                        local-monitoring-policer "local-mon" create
                            description "Monitor for arp, icmp, igmp and all-unspecified"
                            rate packets 100 within 10
                        exit
                        protocol arp create
                            enforcement dynamic "local-mon"
                            dynamic-parameters
                                rate packets 20 within 10
                                exceed-action discard
                            exit
                        exit
                        protocol icmp create
                            enforcement dynamic "local-mon"
                            dynamic-parameters
                                rate packets 20 within 10
                                exceed-action discard
                            exit
                        exit
                        protocol igmp create
                            enforcement dynamic "local-mon"
                            dynamic-parameters
                                rate packets 20 within 10
                                exceed-action discard
                            exit
                        exit
                        protocol all-unspecified create
                            enforcement dynamic "local-mon"
                            dynamic-parameters
                                rate packets 100 within 10
                                exceed-action discard
                            exit
                        exit
                    exit
                exit
            exit
        exit
    exit
    

    For the dcp-dynamic-policy-1 policy configuration:

    • The policy contains no static policers. Per-protocol enforcement policers will be instantiated dynamically but only if triggered by a violation of the local-monitoring-policer.

    • A local-monitoring-policer is configured for the policy. The configured rate determines the rate of arriving protocol packets at which the policy will trigger the automatic instantiation of dynamic per-protocol policers for the interface.

    • Four protocols are configured and they are all associated with the local-monitoring-policer. The all-unspecified protocol will include all other extracted control packets on the interface.

    • Each protocol has its own configured dynamic rates that will be used by the dynamic enforcement policers if they are instantiated. Note these rates are lower than previous scenarios (the within parameter is 10 seconds instead of 1 second).

    • When this DCP policy is associated with an interface, only a single policer (the local-monitoring-policer) will be instantiated (statically/permanently). The per-protocol dynamic policers are only instantiated when there is a violation of the local-monitoring-policer.

    The policy is then associated with the interface:

    *A:R1# configure router interface "int-R1-T1.4.4" 
                            dist-cpu-protection "dcp-dynamic-policy-1"
    *A:R1# 
    
  15. Configure the tester to send:

    • 1pps of ARP

    • 4pps of ICMP

    • 1000pps of IGMP

    Start the tester.

    Figure 4. Dynamic Policing – Local Monitor

    In Dynamic Policing – Local Monitor, the dynamic policers have not been instantiated yet.

  16. The local-monitoring-policer will become non-conforming since the aggregate arrival rate of arp+icmp+igmp+all-unspecified packets is greater than the configured local-monitoring-policer rate of 100 packets within 10 seconds. Dynamic enforcement policers will then be instantiated.

    Figure 5. Dynamic Policers Instantiated

    The ICMP and IGMP dynamic policers will see violations since their dynamic rates are being exceeded.

    *A:R1# tools dump security dist-cpu-protection violators enforcement interface card 4 
    ===============================================================================
    Distributed Cpu Protection Current Interface Enforcer Policer Violators
    ===============================================================================
    Interface                       Policer/Protocol                    Hld Rem       
    -------------------------------------------------------------------------------
    -------------------------------------------------------------------------------
    Violators on Slot-4 Fp-1
    -------------------------------------------------------------------------------
    int-R1-T1.4.4                   icmp                            [D] none         
    int-R1-T1.4.4                   igmp                            [D] none         
    -------------------------------------------------------------------------------
    [S]-Static [D]-Dynamic [M]-Monitor
    -------------------------------------------------------------------------------
    ===============================================================================
    *A:R1#
    

    The ARP and all-unspecified dynamic policers were instantiated but will be counting down their detection time (if this show command is issued within 30 seconds of the attack starting).

    *A:R1# show router interface "int-R1-T1.4.4" dist-cpu-protection 
    
    ===============================================================================
    Interface "int-R1-T1.4.4" (Router: Base)
    ===============================================================================
    Distributed CPU Protection Policy :  dcp-dynamic-policy-1
    
    -------------------------------------------------------------------------------
    Statistics/Policer-State Information
    ===============================================================================
    -------------------------------------------------------------------------------
    Static Policer
    -------------------------------------------------------------------------------
    No entries found
    -------------------------------------------------------------------------------
    -------------------------------------------------------------------------------
    Local-Monitoring Policer
    -------------------------------------------------------------------------------
    Policer-Name        : local-mon 
    Card/FP             : 4/1               Policer-State       : Exceed
    Protocols Mapped    : arp, icmp, igmp, all-unspecified  
    Exceed-Count        : 1249              
    All Dyn-Plcr Alloc. : True              
    -------------------------------------------------------------------------------
    -------------------------------------------------------------------------------
    Dynamic-Policer (Protocol)
    -------------------------------------------------------------------------------
    Protocol(Dyn-Plcr)  : arp 
    Card/FP             : 4/1               Protocol-State      : Conform
    Exceed-Count        : 0                 
    Detec. Time Remain  : 0 seconds         Hold-Down Remain.   : none
    Dyn-Policer Alloc.  : True              
    
    Protocol(Dyn-Plcr)  : icmp 
    Card/FP             : 4/1               Protocol-State      : Exceed
    Exceed-Count        : 72                
    Detec. Time Remain  : 26 seconds        Hold-Down Remain.   : none
    Dyn-Policer Alloc.  : True              
    
    Protocol(Dyn-Plcr)  : igmp 
    Card/FP             : 4/1               Protocol-State      : Exceed
    Exceed-Count        : 56190             
    Detec. Time Remain  : 29 seconds        Hold-Down Remain.   : none
    Dyn-Policer Alloc.  : True              
    
    Protocol(Dyn-Plcr)  : all-unspecified 
    Card/FP             : 4/1               Protocol-State      : Conform
    Exceed-Count        : 0                 
    Detec. Time Remain  : 0 seconds         Hold-Down Remain.   : none
    Dyn-Policer Alloc.  : True              
    -------------------------------------------------------------------------------
    ===============================================================================
    *A:R1# 
    

    After 30 seconds have passed, the ‟Detec. Time Remain” for ARP and all-unspecified will simply read 0 (zero).

    After a few minutes the log events will be collected indicating a non-conformance was seen.

    *A:R1# show log log-id 15 
    
    ===============================================================================
    Event Log 15
    ===============================================================================
    Description : (Not Specified)
    Memory Log contents  [size=1024   next event=10  (not wrapped)]
    
    9 2017/04/27 10:22:53.32 CEST WARNING: SECURITY #2067 Base DCPUPROT
    "Non conformant network_if "int-R1-T1.4.4" on fp 4/1 detected at 04/27/2017 10:18:37.
     Policy "dcp-dynamic-policy-1". Policer="icmp"(dynamic). Excd count=2"
    
    8 2017/04/27 10:22:53.32 CEST WARNING: SECURITY #2067 Base DCPUPROT
    "Non conformant network_if "int-R1-T1.4.4" on fp 4/1 detected at 04/27/2017 10:18:30.
     Policy "dcp-dynamic-policy-1". Policer="igmp"(dynamic). Excd count=80"
    
    ---snip---
    
    *A:R1#
    
  17. Stop the tester.

    The dynamic policer detection timers will start counting down since they are no longer seeing violating packets.

    *A:R1# show router interface "int-R1-T1.4.4" dist-cpu-protection 
    
    ===============================================================================
    Interface "int-R1-T1.4.4" (Router: Base)
    ===============================================================================
    Distributed CPU Protection Policy :  dcp-dynamic-policy-1
    
    -------------------------------------------------------------------------------
    Statistics/Policer-State Information
    ===============================================================================
    -------------------------------------------------------------------------------
    Static Policer
    -------------------------------------------------------------------------------
    No entries found
    -------------------------------------------------------------------------------
    -------------------------------------------------------------------------------
    Local-Monitoring Policer
    -------------------------------------------------------------------------------
    Policer-Name        : local-mon 
    Card/FP             : 4/1               Policer-State       : Exceed
    Protocols Mapped    : arp, icmp, igmp, all-unspecified  
    Exceed-Count        : 5072              
    All Dyn-Plcr Alloc. : True              
    -------------------------------------------------------------------------------
    -------------------------------------------------------------------------------
    Dynamic-Policer (Protocol)
    -------------------------------------------------------------------------------
    Protocol(Dyn-Plcr)  : arp 
    Card/FP             : 4/1               Protocol-State      : Conform
    Exceed-Count        : 0                 
    Detec. Time Remain  : 0 seconds         Hold-Down Remain.   : none
    Dyn-Policer Alloc.  : True              
    
    Protocol(Dyn-Plcr)  : icmp 
    Card/FP             : 4/1               Protocol-State      : Exceed
    Exceed-Count        : 482               
    Detec. Time Remain  : 0 seconds         Hold-Down Remain.   : none
    Dyn-Policer Alloc.  : True              
    
    Protocol(Dyn-Plcr)  : igmp 
    Card/FP             : 4/1               Protocol-State      : Exceed
    Exceed-Count        : 326409            
    Detec. Time Remain  : 0 seconds         Hold-Down Remain.   : none
    Dyn-Policer Alloc.  : True              
    
    Protocol(Dyn-Plcr)  : all-unspecified 
    Card/FP             : 4/1               Protocol-State      : Conform
    Exceed-Count        : 0                 
    Detec. Time Remain  : 0 seconds         Hold-Down Remain.   : none
    Dyn-Policer Alloc.  : True              
    -------------------------------------------------------------------------------
    ===============================================================================
    *A:R1#
    

    After 30 seconds there are no more violators.

    *A:R1# tools dump security dist-cpu-protection violators enforcement interface card 4 
    ===============================================================================
    Distributed Cpu Protection Current Interface Enforcer Policer Violators
    ===============================================================================
    Interface                       Policer/Protocol                    Hld Rem       
    -------------------------------------------------------------------------------
    -------------------------------------------------------------------------------
    Violators on Slot-4 Fp-1
    -------------------------------------------------------------------------------
    -------------------------------------------------------------------------------
    [S]-Static [D]-Dynamic [M]-Monitor
    -------------------------------------------------------------------------------
    ===============================================================================
    *A:R1#
    

    The dynamic policer pool Hi-WaterMark for card 1 fp 1 shows 4 since the highest number of dynamic policers allocated at any one time on the card/fp was 4.

    *A:R1# show card 4 fp 1 dist-cpu-protection 
    
    ===============================================================================
    Card : 4 Forwarding Plane(FP) : 1
    ===============================================================================
    Dynamic Enforcement Policer Pool : 1000
    -------------------------------------------------------------------------------
    
    -------------------------------------------------------------------------------
    Statistics Information
    -------------------------------------------------------------------------------
    Dynamic-Policers Currently In Use      : 0
    Hi-WaterMark Hit Count                 : 4
    Hi-WaterMark Hit Time                  : 04/27/2017 10:10:34 UTC
    Dynamic-Policers Allocation Fail Count : 0
    -------------------------------------------------------------------------------
    ===============================================================================
    *A:R1#
    

    A few minutes later the log events indicate that the flood has ended.

    *A:R1# show log log-id 15 
    
    ===============================================================================
    Event Log 15
    ===============================================================================
    Description : (Not Specified)
    Memory Log contents  [size=1024   next event=12  (not wrapped)]
    
    11 2017/04/27 10:27:53.34 CEST WARNING: SECURITY #2073 Base DCPUPROT
    "Network_if "int-R1-T1.4.4" on fp 4/1 newly conformant at 04/27/2017 10:24:27. Policy
     "dcp-dynamic-policy-1". Policer="igmp"(dynamic). Excd count=326409"
    
    10 2017/04/27 10:27:53.34 CEST WARNING: SECURITY #2073 Base DCPUPROT
    "Network_if "int-R1-T1.4.4" on fp 4/1 newly conformant at 04/27/2017 10:24:27. Policy
     "dcp-dynamic-policy-1". Policer="icmp"(dynamic). Excd count=482"
    
    ---snip---
    
    *A:R1# 
    

Conclusion

Distributed CPU Protection (DCP) offers a powerful rate limiting function for control protocol traffic that is extracted from the data path and sent to the CPM.

This chapter has demonstrated how to configure DCP on an interface and what indications SR OS provides to the operator during a potential attack or misconfiguration.

DCP can also be deployed in scenarios where per-SAP-per-protocol rate limiting is useful, such as for subscriber management in a subscriber per-VLAN scenario. A DCP policy can be assigned to an MSAP policy on a Broadband Network Gateway, for example, to limit traffic related to certain protocols and to discard certain protocols. When deployed in a subscriber management scenario, DCP can help isolate SAPs (subscribers) from each other and even isolate protocols from each other within an individual SAP (subscriber). Many of the same concepts introduced in this chapter are applicable when DCP is deployed in a subscriber management application.