OAM and SAA

This section provides a generalized description of the LSP diagnostics tools. Users should take into account the following restrictions when reading the information contained in this section:

• The 7210 SAS does not support LDP LER ECMP. Descriptions of LDP ECMP as an LER node in LSP diagnostics: LSP ping and trace do not apply to 7210 SAS platforms.

• LDP LSR ECMP is only supported on 7210 SAS-Mxp, 7210 SAS-Sx/S 1/10GE, 7210 SAS-Sx 10/100GE, 7210 SAS-R6, and 7210 SAS-R12. The description on LDP ECMP as LSR node applies only to these platforms.

• 7210 SAS platforms support LDP and BGP 3107 labeled routes. The 7210 SAS does not support LDP FEC stitching to BGP 3107 labeled route to LDP FEC stitching and vice-versa. The following description about stitching of BGP 3107 labeled routes to LDP FEC is provided to describe the behavior in an end-to-end network solution deployed using 7210 SAS and 7x50 nodes, with 7210 SAS nodes acting as the LER node.

The router LSP diagnostics are implementations of LSP ping and LSP trace based on RFC 4379, Detecting Multi-Protocol Label Switched (MPLS) Data Plane Failures. LSP ping provides a mechanism to detect data plane failures in MPLS LSPs. LSP ping and LSP trace are modeled after the ICMP echo request/reply used by ping and trace to detect and localize faults in IP networks.

For a specific LDP FEC, RSVP P2P LSP, or BGP IPv4 Label Router, LSP ping verifies whether the packet reaches the egress label edge router (LER), while in LSP trace mode, the packet is sent to the control plane of each transit label switched router (LSR) which performs various checks to see if it is actually a transit LSR for the path.

The downstream mapping TLV is used in LSP ping and LSP trace to provide a mechanism for the sender and responder nodes to exchange and validate interface and label stack information for each downstream of an LDP FEC or an RSVP LSP and at each hop in the path of the LDP FEC or RSVP LSP.

Two downstream mapping TLVs are supported: the original Downstream Mapping (DSMAP) TLV defined in RFC 4379 and the new Downstream Detailed Mapping (DDMAP) TLV defined in RFC 6424.

When the responder node has multiple equal cost next-hops for an LDP FEC prefix, the downstream mapping TLV can further be used to exercise a specific path of the ECMP set using the path-destination option. The behavior in this case is described in the following ECMP subsection.

MPLS OAM supports segment routing extensions to lsp-ping and lsp-trace as defined in draft-ietf-mpls-spring-lsp-ping.

When the data plane uses MPLS encapsulation, MPLS OAM tools such as lsp-ping and lsp-trace can be used to check connectivity and trace the path to any midpoint or endpoint of an SR-ISIS or SR-OSPF shortest path tunnel.

The CLI options for lsp-ping and lsp-trace are under OAM and SAA for SR-ISIS and SR-OSPF node SID tunnels.

Enter the following OAM command to generate an LSP ping:

oam p2mp-lsp-ping lsp-name [p2mp-instance instance-name [s2l-dest-addr ip-address [...up to 5 max]]] [fc fc-name] [size octets] [ttl label-ttl] [timeout timeout] [detail]

An echo request message is sent on the active P2MP instance and replicated in the datapath over all branches of the P2MP LSP instance. By default, all egress LER nodes that are leaves of the P2MP LSP instance reply to the echo request message.

To reduce the scope of the echo reply message, explicitly enter a list of addresses specifying the egress LER nodes that must reply. A maximum of five addresses can be specified in a single execution of the p2mp-lsp-ping command. If all five egress LER nodes are router nodes, they will parse the list of egress LER addresses and reply. In accordance with RFC 6425, only the top address in the P2MP egress identifier TLV is inspected by an egress LER. When interoperating with other implementations, the egress LER router node responds if its address is in the list. Also, if another vendor implementation is the egress LER, only the egress LER matching the top address in the TLV responds.

If the user enters the same egress LER address more than once in a single p2mp-lsp-ping command, the head-end node displays a response to a single address and displays a single error warning message for the duplicates. When queried over SNMP, the head-end node issues a single response trap and issues no trap for the duplicates.

Set the value of the timeout parameter to the time it would take to get a response from all probed leaves under no failure conditions. For that purpose, the parameter range extends to 120 seconds for a p2mp-lsp-ping from a 10-second lsp-ping for P2P LSP. The default value is 10 seconds.

If the user explicitly lists the address of the egress LER for a specific S2L in the ping command, the router head-end node displays a ‟Send_Fail” error when a specific S2L path is down.

Similarly, if the user explicitly lists the address of the egress LER for a specific S2L in the ping command, the router head-end node displays the timeout error when no response is received for an S2L after the expiry of the timeout timer.

Configure a specific value of the ttl parameter to force the echo request message to expire on a router branch node or a bud LSR node. The bud LSR node replies with a downstream mapping TLV for each branch of the P2MP LSP in the echo reply message. A maximum of 16 downstream mapping TLVs can be included in a single echo reply message. The multipath type is set to zero in each downstream mapping TLV and, consequently, does not include egress address information for the reachable egress LER nodes for this P2MP LSP.

If the router ingress LER node receives the new multipath type field with the list of egress LER addresses in an echo reply message from another vendor implementation, the router ignores the message, but this does not cause a processing error for the downstream mapping TLV.

If the ping command expires at an LSR node that is performing a remerge or crossover operation in the datapath between two or more ILMs of the same P2MP LSP, an echo reply message is generated for each copy of the echo request message received by this node.

If the detail parameter is omitted, the command output provides a high-level summary of error and success codes received.

If the detail parameter is specified, the command output displays a line for each replying node, similar to the output of the LSP ping for a P2P LSP.

The display is delayed until all responses are received or the timer configured in the timeout parameter expires. Entering other CLI commands while waiting for the display is not allowed. Use control-C (^C) to stop the ping operation.

Generate an LSP trace by entering the following OAM command:

oam p2mp-lsp-trace lsp-name p2mp-instance instance-name s2l-dest-addr ip-address [fc fc-name] [size octets] [max-fail no-response-count] [probe-count probes-per-hop] [min-ttl min-label-ttl]] [max-ttl max-label-ttl] [timeout timeout] [interval interval] [detail]

The LSP trace capability allows the user to trace the path of a single S2L path of a P2MP LSP. Its operation is similar to that of the p2mp-lsp-ping command but the sender of the echo reply request message includes the downstream mapping TLV to request the downstream branch information from a branch LSR or bud LSR. The branch LSR or bud LSR will then also include the downstream mapping TLV to report the information about the downstream branches of the P2MP LSP. An egress LER does not include this TLV in the echo response message.

The operation of the probe-count parameter is modeled after the LSP trace on a P2P LSP. It represents the maximum number of probes sent per TTL value before the device gives up on receiving the echo reply message. If a response is received from the traced node before reaching the maximum number of probes, no additional probes are sent for that TTL. The sender of the echo request increments the TTL and uses the information received in the downstream mapping TLV to send probes to the node downstream of the last node that replied. This continues until the egress LER for the traced S2L path replies.

Because the command traces a single S2L path, the timeout and interval parameters keep the same value range as the LSP trace for a P2P LSP.

The following supported options in lsp-trace for P2P LSP are not applicable: path, prefix, path-destination, and [interface | next-hop].

The P2MP LSP trace uses the Downstream Detailed Mapping (DDMAP) TLV defined in RFC 6424. The following figure shows the format of the new DDMAP TLV entered in the path-destination that belongs to one of the possible outgoing interfaces of the FEC.

The DDMAP TLV format is derived from the Downstream Mapping (DSMAP) TLV format. The key change is that in the DDMAP TLV, the variable length and optional fields are converted into sub-TLVs. The fields have the same use and meaning as in RFC 4379.

Similar to P2MP LSP ping, an LSP trace probe results on all egress LER nodes that eventually receive the echo request message, but only the traced egress LER node replies to the last probe.

Any branch LSR node or bud LSR node in the P2MP LSP tree may receive a copy of the echo request message with the TTL in the outer label expiring at this node. However, only a branch LSR or bud LSR that has a downstream branch over which the traced egress LER is reachable must respond.

When a branch LSR or BUD LSR node responds to the sender of the echo request message, it sets the global return code in the echo response message to RC=14, ‟See DDMAP TLV for Return Code and Return Sub-Code” and the return code in the DDMAP TLV corresponding to the outgoing interface of the branch used by the traced S2L path to RC=8, ‟Label switched at stack-depth <RSC>”.

Because a single egress LER address, for example an S2L path, can be traced, the branch LSR or bud LSR node sets the multipath type to zero in the downstream mapping TLV in the echo response message because including an egress LER address is not required.

The 7210 SAS SDP diagnostics are SDP ping and SDP MTU path discovery.

The Nokia service ping feature provides end-to-end connectivity testing for an individual service. Service ping operates at a higher level than the SDP diagnostics in that it verifies an individual service and not the collection of services carried within an SDP.

Service ping is initiated from a 7210 SAS router to verify round-trip connectivity and delay to the far-end of the service. -The Nokia implementation functions for MPLS tunnels and tests the following from edge-to-edge:

• tunnel connectivity

• VC label mapping verification

• service existence

• service provisioned parameter verification

• round trip path verification

• service dynamic configuration verification

While the LSP ping, SDP ping and service ping tools enable transport tunnel testing and verify whether the correct transport tunnel is used, they do not provide the means to test the learning and forwarding functions on a per-VPLS-service basis.

It is conceivable, that while tunnels are operational and correctly bound to a service, an incorrect Forwarding Information Base (FIB) table for a service could cause connectivity issues in the service and not be detected by the ping tools. Nokia has developed VPLS OAM functionality to specifically test all the critical functions on a per-service basis. These tools are based primarily on the IETF document draft-stokes-vkompella-ppvpn-hvpls-oam-xx.txt, Testing Hierarchical Virtual Private LAN Services.

The VPLS OAM tools are:

• MAC ping

  Provides an end-to-end test to identify the egress customer-facing port where a customer MAC was learned. MAC ping can also be used with a broadcast MAC address to identify all egress points of a service for the specified broadcast MAC.

• MAC trace

  Provides the ability to trace a specified MAC address hop-by-hop until the last node in the service domain. An SAA test with MAC trace is considered successful when there is a reply from a far-end node indicating that they have the destination MAC address on an egress SAP or the CPM.

• CPE ping

  Provides the ability to check network connectivity to the specified client device within the VPLS. CPE ping will return the MAC address of the client, as well as the SAP and PE at which it was learned.

• MAC populate

  Allows specified MAC addresses to be injected in the VPLS service domain. This triggers learning of the injected MAC address by all participating nodes in the service. This tool is generally followed by MAC ping or MAC trace to verify if correct learning occurred.

• MAC purge

  Allows MAC addresses to be flushed from all nodes in a service domain.

Ping and trace tools for PWs and LSPs are supported with both IP encapsulation and the MPLS-TP on demand CV channel for non-IP encapsulation (0x025).

The following command provides the debug command for an MPLS-TP tunnel:

tools>dump>router>mpls>tp-tunnel lsp-name[clear]

A:mlstp-dutA# tools dump router mpls tp-tunnel
- tp-tunnel <lsp-name> [clear]
- tp-tunnel id <tunnel-id> [clear]
<lsp-name> : [32 chars max]
<tunnel-id> : [1..61440]
<clear> : keyword - clear stats after reading
*A:mlstp-dutA# tools dump router mpls tp-tunnel "lsp-
"lsp-32" "lsp-33" "lsp-34" "lsp-35" "lsp-36" "lsp-37" "lsp-38" "lsp-39"
"lsp-40" "lsp-41"
*A:mlstp-dutA# tools dump router mpls tp-tunnel "lsp-32"
Idx: 1-32 (Up/Up): pgId 4, paths 2, operChg 1, Active: Protect
TunnelId: 42::0.0.3.233::32-42::0.0.3.234::32
PgState: Dn, Cnt/Tm: Dn 1/000 04:00:48.160 Up:3/000 00:01:25.840
MplsMsg: tpDn 0/000 00:00:00.000, tunDn 0/000 00:00:00.000
wpDn 0/000 00:00:00.000, ppDn 0/000 00:00:00.000
wpDel 0/000 00:00:00.000, ppDel 0/000 00:00:00.000
tunUp 1/000 00:00:02.070
Paths:
Work (Up/Dn): Lsp 1, Lbl 32/32, If 2/128 (1/2/3 : 0.0.0.0)
Tmpl: ptc: , oam: privatebed-oam-template (bfd: privatebed-bfd-template(np)-10 ms)
Bfd: Mode CC state Dn/Up handle 160005/0
Bfd-CC (Cnt/Tm): Dn 1/000 04:00:48.160 Up:1/000 00:01:23.970
State: Admin Up (1::1::1) port Up , if Dn , operChg 2
DnReasons: ccFault ifDn
Protect (Up/Up): Lsp 2, Lbl 2080/2080, If 3/127 (5/1/1 : 0.0.0.0)
Tmpl: ptc: privatebed-protection-template, oam: privatebed-oam-
template (bfd: privatebed-
bfd-template(np)-10 ms)
Bfd: Mode CC state Up/Up handle 160006/0
Bfd-CC (Cnt/Tm): Dn 0/000 00:00:00.000 Up:1/000 00:01:25.410
State: Admin Up (1::1::1) port Up , if Up , operChg 1
Aps: Rx - 5, raw 3616, nok 0(), txRaw - 3636, revert Y
Pdu: Rx - 0x1a-21::0101 (SF), Tx - 0x1a-21::0101 (SF)
State: PF:W:L LastEvt pdu (L-SFw/R-SFw)
Tmrs: slow
Defects: None Now: 000 05:02:19.130
Seq Event state TxPdu RxPdu Dir Act Time
=== ====== ======== ========== ========== ===== ==== ================
000 start UA:P:L SF (0,0) NR (0,0) Tx--> Work 000 00:00:02.080
001 pdu UA:P:L SF (0,0) SF (0,0) Rx<-- Work 000 00:01:24.860
002 pdu UA:P:L SF (0,0) NR (0,0) Rx<-- Work 000 00:01:26.860
003 pUp NR NR (0,0) NR (0,0) Tx--> Work 000 00:01:27.440
004 pdu NR NR (0,0) NR (0,0) Rx<-- Work 000 00:01:28.760
005 wDn PF:W:L SF (1,1) NR (0,0) Tx--> Prot 000 04:00:48.160
006 pdu PF:W:L SF (1,1) NR (0,1) Rx<-- Prot 000 04:00:48.160
007 pdu PF:W:L SF (1,1) SF (1,1) Rx<-- Prot 000 04:00:51.080

A:SASR1# /tools dump router mpls  mpls-tp check-lbl-range 
  - mpls-tp check-lbl-range <range1> <range2>

<check-lbl-range>    : keyword
<range1>             : [32..65520]
<range2>             : [32..65520]

*A:7210SAS# /debug service id 700 sdp 200:700 event-type ?{config-change|oper-
status-change|neighbor-discovery|control-channel-status}

*A:7210SAS# /debug service id 700 sdp 200:700 event-type control-channel-status 

*A:7210SAS# 
1 2012/08/31 09:56:12.09 EST MINOR: DEBUG #2001 Base PW STATUS SIG PKT (RX):
PW STATUS SIG PKT (RX):: 
Sdp Bind 200:700 Instance 3
    Version          : 0x0
    PW OAM Msg Type  : 0x27
    Refresh Time     : 0xa
    Total TLV Length : 0x8
    Flags            : 0x0
    TLV Type         : 0x96a
    TLV Len          : 0x4
    PW Status Bits   : 0x0

 
2 2012/08/31 09:56:22.09 EST MINOR: DEBUG #2001 Base PW STATUS SIG PKT (RX):
PW STATUS SIG PKT (RX):: 
Sdp Bind 200:700 Instance 3
    Version          : 0x0
    PW OAM Msg Type  : 0x27
    Refresh Time     : 0xa
    Total TLV Length : 0x8
    Flags            : 0x0
    TLV Type         : 0x96a
    TLV Len          : 0x4
    PW Status Bits   : 0x0

 
3 2012/08/31 09:56:29.44 EST MINOR: DEBUG #2001 Base PW STATUS SIG PKT (TX):
PW STATUS SIG PKT (TX):: 
Sdp Bind 200:700 Instance 3
    Version          : 0x0
    PW OAM Msg Type  : 0x27
    Refresh Time     : 0x1e
    Total TLV Length : 0x8
    Flags            : 0x0
    TLV Type         : 0x96a
    TLV Len          : 0x4
    PW Status Bits   : 0x0

Two-Way Active Measurement Protocol (TWAMP) provides a standards-based method for measuring the round-trip IP performance (packet loss, delay and jitter) between two devices. TWAMP uses the methodology and architecture of One-Way Active Measurement Protocol (OWAMP) to define a way to measure two-way or round-trip metrics.

There are four logical entities in TWAMP:

• the control-client

• the session-sender

• the server

• the session-reflector

The control-client and session-sender are typically implemented in one physical device (the ‟client”) and the server and session-reflector in a second physical device (the ‟server”) with which the two-way measurements are being performed. The 7210 SAS acts as the server. The control-client and server establishes a TCP connection and exchange TWAMP-Control messages over this connection. When the control-client requires to start testing, the client communicates the test parameters to the server. If the server corresponds to conduct the described tests, the test begins as soon as the client sends a Start-Sessions message. As part of a test, the session sender sends a stream of UDP-based test packets to the session-reflector, and the session reflector responds to each received packet with a response UDP-based test packet. When the session-sender receives the response packets from the session-reflector, the information is used to calculate two-way delay, packet loss, and packet delay variation between the two devices.

TWAMP Light is an optional model included in the TWAMP standard RFC5357 that uses standard TWAMP test packets but provides a lightweight approach to gathering ongoing IP delay performance data for base router and per-VPRN statistics. Full details are described in Appendix I of RFC 5357 (Active Two Way Measurement Protocol). The 7210 SAS implementation supports the TWAMP Light model for gathering delay and loss statistics.

For TWAMP Light, the TWAMP Client/Server model is replaced with the Session Controller/Responder model. In general terms, the Session Controller is the launch point for the test packets and the Responder performs the reflection function.

TWAMP Light maintains the TWAMP test packet exchange but eliminates the TWAMP TCP control connection with local configurations; however, not all negotiated control parameters are replaced with local configuration. For example, CoS parameters communicated over the TWAMP control channel are replaced with a reply-in-kind approach. The reply-in-kind model reflects back the received CoS parameters, which are influenced by the reflector’s QoS policies.

The responder function is configured under the config>router>twamp-light command hierarchy for base router reflection, and under the config>service>vprn>twamp-light command hierarchy for per VPRN reflection. The TWAMP Light reflector function is configured per context and must be activated before reflection can occur; the function is not enabled by default for any context. The reflector requires the operator to define the TWAMP Light UDP listening port that identifies the TWAMP Light protocol and the prefixes that the reflector will accept as valid sources for a TWAMP Light request. If the configured TWAMP Light listening UDP port is in use by another application on the system, a Minor OAM message will be presented indicating that the port is unavailable and that the activation of the reflector is not allowed.

f the source IP address in the TWAMP Light packet arriving on the responder does not match a configured IP address prefix, the packet is dropped. Multiple prefix entries may be configured per context on the responder. An inactivity timeout under the config>oam-test>twamp>twamp-light hierarchy defines the amount of time the reflector will keep the individual reflector sessions active in the absence of test packets. A responder requires CPM3 or better hardware.

TWAMP Light test packet launching is controlled by the OAM Performance Monitoring (OAM-PM) architecture and adheres to those rules; this includes the assignment of a test Id. TWAMP Light does not carry the 4-byte test ID in the packet to remain locally significant and uniform with other protocols under the control of the OAM-PM architecture. The OAM-PM construct allow the various test parameters to be defined. These test parameters include the IP session-specific information which allocates the test to the specific routing instance, the source and destination IP address, the destination UDP port (which must match the listening UDP port on the reflector) and a number of other options that allow the operator to influence the packet handling. The probe interval and padding size can be configured under the specific session. The size of the all ‟0” padding can be included to ensure that the TWAMP packet is the same size in both directions. The TWAMP PDU definition does not accomplish symmetry by default. A pad size of 27 bytes will accomplish symmetrical TWAMP frame sizing in each direction.

The OAM-PM architecture does not perform any validation of the session information. The test will be allowed to be activated regardless of the validity of this information. For example, if the configured source IP address is not local within the router instance to which the test is allocated, the test will start sending TWAMP Light packets but will not receive any responses.

The OAM Performance Monitoring (OAM-PM) section of this guide provides more information describing the integration of TWAMP Light and the OAM-PM architecture, including hardware dependencies.

The following is a summary of supported TWAMP Light functions.

• base router instances for network interfaces and IES services:

  • IPv6 addresses are supported only with IP interfaces that support IPv6, such as an IES IP interface in access-uplink and network mode. IPv6 is not supported for IP interfaces that do not support IPv6; for example, routed VPLS services in access-uplink mode do not support IPv6, and therefore TWAMP Light IPv6 sessions are not supported with it.

• per-VPRN service context

• IPv4 and IPv6:

  • must be unicast

  • for IPv6, addresses cannot be a reserved or link local address

• reflector prefix definition for acceptable TWAMP Light sources:

  • Prefix list may be added and removed without shutting down the reflector function.

  • If no prefixes are defined, the reflector will drop all TWAMP Light packets.

• integration with OAM-PM architecture capturing delay and loss measurement statistics:

  • Not available from interactive CLI.

  • Multiple test sessions can be configured between the same source and destination IP endpoints. The tuple of Source IP, Destination IP, Source UDP, and Destination UDP provide a unique index for each test.

The following example shows a basic configuration using TWAMP Light to monitor two IP endpoints in a VPRN, including the default TWAMP Light values that were not overridden with configuration entries.

config>test-oam>twamp>twamp-light# info detail
--------------------------------------------------------------------------
(default)      inactivity-timeout 100 
-------------------------------------------------------------------------- 

config>service>vprn# info
--------------------------------------------------------------------------- 
            route-distinguisher 65535:500
            auto-bind ldp
            vrf-target target:65535:500
            interface "to-cpe31" create
                address 10.1.1.1/30
                sap 1/1/2:500 create
                exit
            exit
            static-route 192.168.1.0/24 next-hop 10.1.1.2
            bgp
                no shutdown
            exit
            twamp-light
                reflector udp-port 15000 create
                    description "TWAMP Light reflector VPRN 500"
                    prefix 10.2.1.1/32 create
                        description "Process only 10.2.1.1 TWAMP Light Packets"
                    exit
                    prefix 172.16.1.0/24 create
                        description "Process all 172.16.1.0 TWAMP Light packets"
                    exit
                    no shutdown
                exit
            exit
            no shutdown
------------------------------------------------------------------------------

config>service>vprn# info
-------------------------------------------------------------------------
            route-distinguisher 65535:500
            auto-bind ldp
            vrf-target target:65535:500
            interface "to-cpe28" create
                address 10.2.1.1/30
                sap 1/1/4:500 create
                exit
            exit
            static-route 192.168.2.0/24 next-hop 10.2.1.2
            no shutdown
------------------------------------------------------------------------- 

config>oam-pm>session# info detail
-------------------------------------------------------------------------
            bin-group 2
            meas-interval 15-mins create
                intervals-stored 8
            exit
            ip
                dest-udp-port 15000
                destination 10.1.1.1
                fc "l2"
(default)           no forwarding 
                profile in
                router 500
                source 10.2.1.1
(default)          ttl 255  
                twamp-light test-id 500 create
(default)              interval 100
(default)              pad-size 0
(default)              no test-duration
                   no shutdown
                exit
            exit
------------------------------------------------------------------------- 

The IEEE and the ITU-T use their own nomenclature when describing administrative contexts and functions. This introduces a level of complexity to configuration, description and different vendors naming conventions. The 7210 SAS OS CLI has chosen to standardize on the IEEE 802.1ag naming where overlap exists. ITU-T naming is used when no equivalent is available in the IEEE standard. In the following definitions, both the IEEE name and ITU-T names are provided for completeness, using the format IEEE Name/ITU-T Name.

Maintenance Domain (MD)/Maintenance Entity (ME) is the administrative container that defines the scope, reach and boundary for faults. It is typically the area of ownership and management responsibility. The IEEE allows for various formats to name the domain, allowing up to 45 characters, depending on the format selected. ITU-T supports only a format of none and does not accept the IEEE naming conventions:

• 0

  Undefined and reserved by the IEEE.

• 1

  No domain name. It is the only format supported by Y.1731 as the ITU-T specification does not use the domain name. This is supported in the IEEE 802.1ag standard but not in currently implemented for 802.1ag defined contexts.

• 2,3,4

  Provides the ability to input various different textual formats, up to 45 characters. The string format (2) is the default and therefore the keyword is not shown when looking at the configuration.

Maintenance Association (MA)/Maintenance Entity Group (MEG) is the construct where the different management entities will be contained. Each MA is uniquely identified by its MA-ID. The MA-ID is comprised of the by the MD level and MA name and associated format. This is another administrative context where the linkage is made between the domain and the service using the bridging-identifier configuration option. The IEEE and the ITU-T use their own specific formats. The MA short name formats (0-255) have been divided between the IEEE (0-31, 64-255) and the ITU-T (32-63), with five currently defined (1-4, 32). Even though the different standards bodies do not have specific support for the others formats a Y.1731 context can be configured using the following IEEE format options:

• 1 (Primary VID) - values 0 to 4094

• 2 (String) - raw ASCII, excluding 0-31 decimal/0-1F hex (which are control characters) form the ASCII table

• 3 (2-octet integer) - 0 to 65535

• 4 (VPN ID) - hex value as described in RFC 2685, Virtual Private Networks Identifier

• 32 (icc-format) - exactly 13 characters from the ITU-T recommendation T.50

Maintenance Domain Level (MD Level)/Maintenance Entity Group Level (MEG Level) is the numerical value (0-7) representing the width of the domain. The wider the domain, higher the numerical value, the farther the ETH-CFM packets can travel. It is important to understand that the level establishes the processing boundary for the packets. Strict rules control the flow of ETH-CFM packets and are used to ensure correct handling, forwarding, processing and dropping of these packets. To keep it simple ETH-CFM packets with higher numerical level values will flow through MEPs on MIPs on SAPs configured with lower level values. This allows the operator to implement different areas of responsibility and nest domains within each other. Maintenance association (MA) includes a set of MEPs, each configured with the same MA-ID and MD level used verify the integrity of a single service instance.

Maintenance Endpoint (MEP)/MEG Endpoint (MEP) are the workhorses of ETH-CFM. A MEP is the unique identification within the association (0-8191). Each MEP is uniquely identified by the MA-ID, MEPID tuple. This management entity is responsible for initiating, processing and terminating ETH-CFM functions, following the nesting rules. MEPs form the boundaries which prevent the ETH-CFM packets from flowing beyond the specific scope of responsibility. A MEP has direction, up or down. Each indicates the directions packets will be generated; UP toward the switch fabric, down toward the SAP away from the fabric. Each MEP has an active and passive side. Packets that enter the active point of the MEP will be compared to the existing level and processed accordingly. Packets that enter the passive side of the MEP are passed transparently through the MEP. Each MEP contained within the same maintenance association and with the same level (MA-ID) represents points within a single service. MEP creation on a SAP is allowed only for Ethernet ports with NULL, Q-tags, Q-in-Q encapsulations. MEPs may also be created on SDP bindings.

Maintenance Intermediate Point (MIP)/MEG Intermediate Point (MIP) are management entities between the terminating MEPs along the service path. These provide insight into the service path connecting the MEPs. MIPs only respond to Loopback Messages (LBM) and Linktrace Messages (LTM). All other CFM functions are transparent to these entities. Only one MIP is allowed per SAP or SDP. The creation of the MIPs can be done when the lower level domain is created (explicit). This is controlled by the use of the mhf-creation mode within the association under the bridge-identifier. MIP creation is supported on a SAP and SDP, not including Mesh SDP bindings. By default, no MIPs are created.

There are two locations in the configuration where ETH-CFM is defined. The domains, associations (including linkage to the service id), MIP creation method, common ETH-CFM functions and remote MEPs are defined under the top level eth-cfm command. It is important to note, when Y.1731 functions are required the context under which the MEPs are configured must follow the Y.1731 specific formats (domain format of none, MA format icc-format). When these parameters have been entered, the MEP and possibly the MIP can be defined within the service under the SAP or SDP.

ETH-CFM support matrix for the 7210 SAS-T (network mode), ETH-CFM support matrix for the 7210 SAS-T (access-uplink mode), ETH-CFM support matrix for 7210 SAS-Mxp devices, ETH-CFM support matrix for 7210 SAS-R6 and 7210 SAS-R12 devices, ETH-CFM support matrix for 7210 SAS-Sx/S 1/10GE devices, and ETH-CFM support matrix for 7210 SAS-Sx 10/100GE devices are general tables that indicate the ETH-CFM support for the different services and endpoints. It is not meant to indicate the services that are supported or the requirements for those services on the individual platforms.

The following figures show the detailed IEEE representation of MEPs, MIPs, levels and associations, using the standards defined icons.

Accurate results for one-way and two-way delay measurement tests using Y.1731 messages are obtained if the nodes are capable of time stamping packets in hardware:

• 7210 SAS-Sx 10/100GE support is as follows:

  • Y.1731 2-DM messages for both Down MEPs and UP MEPs, 1-DM for both Down MEPs and UP MEPs, and 2-SLM for both Down MEPs and UP MEPs use software-based timestamps on Tx and hardware based timestamp on Rx. It uses the system clock (free-running or synchronized to NTP) to obtain the timestamps.

• 7210 SAS-T (network mode), 7210 SAS-Sx 1/10GE, 7210 SAS-Mxp, 7210 SAS-R6 and 7210 SAS-R12 support is as follows:

  • Y.1731 2-DM messages for both Down MEPs and UP MEPs, 1-DM for both Down MEPs and UP MEPs, and 2-SLM for both Down MEPs and UP MEPs use software-based timestamps on transmission and hardware-based timestamps on receipt. The timestamps are obtained as follows:

  • from NTP, when NTP is enabled and PTP is disabled

  • from PTP, when PTP is enabled (irrespective of whether NTP is disabled or enabled)

  • from PTP, when PTP is enabled and NTP is configured to use PTP for system time

  • from free-running system time, when both NTP and PTP are disabled.

• The 7210 SAS-T (access-uplink mode) support is as follows:

  • Y.1731 2-DM messages for Down MEPs uses hardware timestamps for both Rx (packets received by the node) and Tx (packets sent out of the node). The timestamps is obtained from a free-running hardware clock. It provides accurate 2-way delay measurements and it is not recommended to use it for computing 1-way delay.

  • Y.1731 2-DM messages for UP MEPs, 1-DM for both Down MEPs and UP MEPs, and 2-SLM for both Down MEPs and UP MEPs use software based timestamps on Tx and hardware based timestamp on Rx. The timestamps are obtained as follows:

  • from NTP, when NTP is enabled and PTP is disabled

  • from PTP, when PTP is enabled (irrespective of whether NTP is disabled or enabled)

  • from PTP, when PTP is enabled and NTP is configured to use PTP for system time

  • from free-running system time, when both NTP and PTP are disabled.

The Ethernet Bandwidth Notification (ETH-BN) function is used by a server MEP to signal link bandwidth changes to a client MEP.

This functionality is for point-to-point microwave radios. When a microwave radio uses adaptive modulation, the capacity of the radio can change based on the condition of the microwave link. For example, in adverse weather conditions that cause link degradation, the radio can change its modulation scheme to a more robust one (which will reduce the link bandwidth) to continue transmitting.

This change in bandwidth is communicated from the server MEP on the radio, using an Ethernet Bandwidth Notification Message (ETH-BNM), to the client MEP on the connected router. The server MEP transmits periodic frames with ETH-BN information, including the interval, the nominal and currently available bandwidth. A port MEP with the ETH-BN feature enabled will process the information contained in the CFM PDU and appropriately adjust the rate of traffic sent to the radio.

A port MEP that is not a LAG member port supports the client side reception and processing of the ETH-BN CFM PDU sent by the server MEP. By default, processing is disabled. The config>port>ethernet>eth-cfm>mep eth-bn>receive CLI command sets the ETH-BN processing state on the port MEP. A port MEP supports untagged packet processing of ETH-CFM PDUs at domain levels 0 and 1 only. The port client MEP sends the ETH-BN rate information received to be applied to the port egress rate in a QoS update. A pacing mechanism limits the number of QoS updates sent. The config>port>ethernet>eth-cfm>mep>eth-bn>rx-update-pacing CLI command allows the updates to be paced using a configurable range of 1 to 600 seconds (the default is 5 seconds). The pacing timer begins to count down following the most recent QoS update sent to the system for processing. When the timer expires, the most recent update that arrived from the server MEP is compared to the most recent value sent for system processing. If the value of the current bandwidth is different from the previously processed value, the update is sent and the process begins again. Updates with a different current bandwidth that arrive when the pacing timer has already expired are not subject to a timer delay. See the 7210 SAS-Mxp, R6, R12, S, Sx, T Interface Configuration Guide for more information about these CLI commands.

A complimentary QoS configuration is required to allow the system to process current bandwidth updates from the CFM engine. The config>port>ethernet>eth-bn-egress-rate-changes CLI command is required to enable the QoS function to update the port egress rates based on the current available bandwidth updates from the CFM engine. By default, the function is disabled.

Both the CFM and QoS functions must be enabled for the changes in current bandwidth to dynamically update the egress rate.

When the MEP enters a state that prevents it from receiving the ETH-BNM, the current bandwidth last sent for processing is cleared and the egress rate reverts to the configured rate. Under these conditions, the last update cannot be guaranteed as current. Explicit notification is required to dynamically update the port egress rate. The following types of conditions lead to ambiguity:

• administrative MEP shut down

• port admin down

• port link down

• eth-bn no receive transitioning the ETH-BN function to disable

If the eth-bn-egress-rate-changes command is disabled using the no option, CFM continues to send updates, but the updates are held without affecting the port egress rate.

The ports supporting ETH-BN MEPs can be configured for the network, access, hybrid, and access-uplink modes. When ETH-BN is enabled on a port MEP and the config>port>ethernet>eth-cfm>mep>eth-bn>receive and the QoS config>port>ethernet>eth-bn-egress-rate-changes contexts are configured, the egress rate is dynamically changed based on the current available bandwidth indicated by the ETH-BN server.

The port egress rate is capped by the minimum of the configured egress-rate, and the maximum port rate. The minimum egress rate using ETH-BN is 1024 kb/s. If a current bandwidth of zero is received, it does not affect the egress port rate and the previously processed current bandwidth will continue to be used.

The client MEP requires explicit notification of changes to update the port egress rate. The system does not timeout any previously processed current bandwidth rates using a timeout condition. The specification does allow a timeout of the current bandwidth if a frame has not been received in 3.5 times the ETH-BNM interval. However, the implicit approach can lead to misrepresented conditions and has not been implemented.

When you start or restart the system, the configured egress rate is used until an ETH-BNM arrives on the port with a new bandwidth request from the ETH-BN server MEP.

An event log is generated each time the egress rate is changed based on reception of a BNM. If a BNM is received that does not result in a bandwidth change, no event log is generated.

The destination MAC address can be a Class 1 multicast MAC address (that is, 01-80-C2-00-0x) or the MAC address of the port MEP configured. Standard CFM validation and identification must be successful to process CFM PDUs.

For information about the eth-bn-egress-rate-changes command, see the 7210 SAS-Mxp, R6, R12, S, Sx, T Interface Configuration Guide.

The Bandwidth Notification Message (BNM) PDU used for ETH-BN information is a sub-OpCode within the Ethernet Generic Notification Message (ETH-GNM).

The following table shows the BNM PDU format fields.

The show eth-cfm mep eth-bandwidth-notification display output includes the ETH-BN values received and extracted from the PDU, including a last reported value and the pacing timer. If the n/a value appears in the field, it indicates that field has not been processed.

The base show eth-cfm mep output is expanded to include the disposition of the ETH-BN receive function and the configured pacing timer.

The show port port-id detail is expanded to include an Ethernet Bandwidth Notification Message Information section. This section includes the ETH-BN Egress Rate disposition and the current Egress BN rate being used.

The 7210 SAS supports port-based MEPs for use with CFM ETH-BN. The port MEP must be configured at level 0 or 1 and can be used for ETH-BN message reception and processing as described in ITU-T Y.1731 Ethernet Bandwidth Notification. Port-based MEPs only support CFM CC, LT, LS, and RDI message processing; other CFM and Y.1731 messages are not supported.

A number of statistics are available to view the current processing requirements for CFM. Any packet that is counted against the CFM resource is included in the statistics counters. The counters do not include sub-second CCM and ETH-CFM PDUs generated by non-ETH-CFM functions (which include OAM-PM and SAA) or filtered by a security configuration.

SAA and OAM-PM use standard CFM PDUs. The reception of these packets is included in the receive statistics. However, SAA and OAM-PM launch their own test packets and do not consume ETH-CFM transmission resources.

Per-system and per-MEP statistics are included with a per-OpCode breakdown. These statistics help operators determine the busiest active MEPs on the system and provide a breakdown of per-OpCode processing at the system and MEP level.

Use the show eth-cfm statistics command to view the statistics at the system level. Use the show eth-cfm mep mep-id domain md-index association ma-index statistics command to view the per-MEP statistics. Use the clear eth-cfm mep mep-id domain md-index association ma-index statistics command to clear statistics. The clear command clears the statistics for only the specified function. For example, clearing the system statistics does not clear the individual MEP statistics because each MEP maintains its own unique counters.

All known OpCodes are listed in the transmit and receive columns. Different versions for the same OpCode are not displayed. This does not imply that the network element supports all functions listed in the table. Unknown OpCodes are dropped.

Use the tools dump eth-cfm top-active-meps command to display the top ten active MEPs in the system. This command provides a nearly real-time view of the busiest active MEPS by displaying the active (not shutdown) MEPs and inactive (shutdown) MEPs in the system. ETH-CFM MEPs that are shutdown continue to consume CPM resources because the main task is syncing the PDUs. The counts begin from the last time that the command was issued using the clear option.

tools dump eth-cfm top-active-meps
Calculating most active MEPs in both direction without clear ...

MEP                  Rx Stats     Tx Stats     Total Stats
-------------------- ------------ ------------ ------------
12/4/28              3504497      296649       3801146
14/1/28              171544       85775        257319
14/2/28              150942       79990        230932

tools dump eth-cfm top-active-meps clear
Calculating most active MEPs in both direction with clear ...

MEP                  Rx Stats     Tx Stats     Total Stats
-------------------- ------------ ------------ ------------
12/4/28              3504582      296656       3801238
14/1/28              171558       85782        257340
14/2/28              150949       79997        230946

tools dump eth-cfm top-active-meps clear
Calculating most active MEPs in both direction with clear ...

MEP                  Rx Stats     Tx Stats     Total Stats
-------------------- ------------ ------------ ------------
12/4/28              28           2            30
14/1/28              5            2            7
14/2/28              3            2            5 

The following lists ETH-CFM configuration guidelines:

• Up MEPs and bidirectional MIPs are not created by default on system bootup, and additional resources must be allocated to enable Up MEP and bidirectional MIP functionality. By default, no resources are allocated. Before Up MEPs and bidirectional MIPs can be created, the user must first use the configure>system>resource-profile context to explicitly allocate hardware resources for use with these features. The software will reject the configuration to create an Up MEP or bidirectional MIP and generate an error until resources are allocated. See the 7210 SAS-Mxp, R6, R12, S, Sx, T Basic System Configuration Guide for more information.

• 7210 SAS platforms support functionality for ingress and egress MIPs. In most services, only ingress MIPs are supported. Some services also support both ingress and egress MIPs, also called bidirectional MIPs. An ingress MIP or a Down MIP processes messages in the ingress direction when the OAM message is received on ingress of the SAP or port (subject to other checks). An egress MIP or an UP MIP refers to a MIP that processes messages in the egress direction when the OAM message is being sent out of the SAP/port. See ETH-CFM support matrix for the 7210 SAS-T (network mode), ETH-CFM support matrix for the 7210 SAS-T (access-uplink mode), ETH-CFM support matrix for 7210 SAS-Mxp devices, ETH-CFM support matrix for 7210 SAS-R6 and 7210 SAS-R12 devices, ETH-CFM support matrix for 7210 SAS-Sx/S 1/10GE devices, and ETH-CFM support matrix for 7210 SAS-Sx 10/100GE devices for more information about ingress MIP, egress MIP, and bidirectional MIP support for service entities.

• On 7210 SAS platforms, Ethernet Linktrace Response (ETH-LTR) is always sent out with priority 7.

• 7210 SAS platforms, send out all CFM packets as in-profile. Currently, there is no mechanism in the SAA tools to specify the profile of the packet.

• On the 7210 SAS-R6 and 7210 SAS-R12, and on the 7210 SAS-Sx/S 1/10GE operating in the standalone-VC mode, before configuring bidirectional MIPs for an Epipe SAP or Epipe SDP binding, resources must be allocated to both Down MEPs and Up MEPs. That is, bidirectional MIPs in an Epipe service use the resources from both the Down MEP and Up MEP resource pools. By default, no resources are allocated for bidirectional MIPs, and configuration attempts before resource allocation are not permitted by the system and will generate a system error/log. See the 7210 SAS-Mxp, R6, R12, S, Sx, T Basic System Configuration Guide for more information.

• Sender ID TLV processing (insertion and reception) is not supported for CCM messages for MEPs that are implemented in hardware; that is, on 7210 SAS-T Down MEPs in access-uplink mode. For these MEPs, Sender ID TLV processing is supported only for LTM and LBM messages.

• Sender ID TLV processing is supported only for service MEPs. It is not supported for G.8032 MEPs.

• Facility MEPs are not supported on the 7210 SAS. G8032 MEPs are supported on the 7210 SAS.

• Ethernet rings are not configurable under all service types. Any service restrictions for MEP direction or MIP support will override the generic capability of the Ethernet ring MPs. See the 7210 SAS-Mxp, R6, R12, S, Sx, T Interface Configuration Guide for more information about Ethernet rings.

• On 7210 SAS devices, when two bidirectional MIPs are configured in an Epipe service on both the service entities and endpoints (for example, on both the SAP and SDP configured in the Epipe service), only the MIP ingressing in the direction of linktrace messages responds. This is applicable to 7210 SAS platforms that support both ingress and egress MIPs (also referred to as bidirectional MIPs).

CFM MEP declares a connectivity fault when its defect flag is equal to or higher than its configured lowest defect priority. The defect can be any of the following depending on configuration:

• DefRDICCM

• DefMACstatus

• DefRemoteCCM

• DefErrorCCM

• DefXconCCM

The following additional fault condition applies to Y.1731 MEPs:

• reception of AIS for the local MEP level

Setting the lowest defect priority to allDef may cause problems when fault propagation is enabled in the MEP. In this scenario, when MEP A sends CCM to MEP B with interface status down, MEP B will respond with a CCM with RDI set. If MEP A is configured to accept RDI as a fault, then it gets into a dead lock state, where both MEPs will declare fault and never be able to recover.

The default lowest defect priority is DefMACstatus, which will not be a problem when interface status TLV is used. It is also very important that different Ethernet OAM strategies should not overlap the span of each other. In some cases, independent functions attempting to perform their normal fault handling can negatively impact the other. This interaction can lead to fault propagation in the direction toward the original fault, a false positive, or worse, a deadlock condition that may require the operator to modify the configuration to escape the condition. For example, overlapping Link Loss Forwarding (LLF) and ETH-CFM fault propagation could cause these issues.

For the DefRemoteCCM fault, it is raised when any remote MEP is down. So whenever a remote MEP fails and fault propagation is enabled, a fault is propagated to SMGR.

When CFM is the OAM module at the other end, it is required to use any of the following methods (depending on local configuration) to notify the remote peer:

• Generating AIS for certain MEP levels

• Sending CCM with interface status TLV ‟down”

• Stopping CCM transmission

User can configure UP MEPs to use Interface Status TLV with fault propagation. Special considerations apply only to Down MEPs.

When a fault is propagated by the service manager, if AIS is enabled on the SAP/SDP-binding, then AIS messages are generated for all the MEPs configured on the SAP/SDP-binding using the configured levels.

Note that the existing AIS procedure still applies even when fault propagation is disabled for the service or the MEP. For example, when a MEP loses connectivity to a configured remote MEP, it generates AIS if it is enabled. The new procedure that is defined in this document introduces a new fault condition for AIS generation, fault propagated from SMGR, that is used when fault propagation is enabled for the service and the MEP.

The transmission of CCM with interface status TLV must be done instantly without waiting for the next CCM transmit interval. This rule applies to CFM fault notification for all services.

Notifications from SMGR to the CFM MEPs for fault propagation should include a direction for the propagation (up or down: up means in the direction of coming into the SAP/SDP-binding; down means in the direction of going out of the SAP/SDP-binding), so that the MEP knows what method to use. For instance, an up fault propagation notification to a down MEP will trigger an AIS, while a down fault propagation to the same MEP can trigger a CCM with interface TLV with status down.

For a specific SAP/SDP-binding, CFM and SMGR can only propagate one single fault to each other for each direction (up or down).

When there are multiple MEPs (at different levels) on a single SAP/SDP-binding, the fault reported from CFM to SMGR will be the logical OR of results from all MEPs. Basically, the first fault from any MEP will be reported, and the fault will not be cleared as long as there is a fault in any local MEP on the SAP/SDP-binding.

Down and up MEPs are supported for Epipe services, as well as fault propagation. When there are both up and down MEPs configured in the same SAP/SDP-binding and both MEPs have fault propagation enabled, a fault detected by one of them will be propagated to the other, which in turn will propagate fault in its own direction.

A fault on the access-uplink port brings down all access ports with services independent of the encapsulation type of the access port (null, dot1q, or QinQ), that is, support Link Loss Forwarding (LLF). A fault propagated from the access-uplink port to access ports is based on configuration. A fault is propagated only in a single direction from the access-uplink port to access port.

A fault on the access-uplink port is detected using Loss of Signal (LoS) and EFM-OAM.

The following figure shows local fault propagation.

A fault on the uplink port or LAG brings down all access ports with services independent of the encapsulation type of the access port (null, dot1q, or QinQ), that is, support Link Loss Forwarding (LLF). A fault propagated from the uplink port or LAG to access ports is based on configuration. A fault is propagated only in a single direction from the uplink port or LAG to access port.

A fault on the uplink port or LAG is detected using Loss of Signal (LoS) and EFM-OAM.

The following figure show local fault propagation.

SAA allows two-way timing for several applications. This provides the carrier and their customers with data to verify that the SLA agreements are being correctly enforced.

This section describes the prerequisites for using the testhead tool.

This section provides the configuration guidelines for the testhead OAM tool. It is applicable to all the platforms described in this guide unless a specific platform is called out explicitly:

• On the 7210 SAS-Sx 10/100GE platform, the following limitation must be addressed in the testhead configuration: the testhead loopback port and MAC swap loopback port must be configured with the same Ethernet speeds and have the same operational speeds. Failure to do so will cause the testhead session start to fail.

• SAPs configured on LAG cannot be configured for testing with testhead tool. Other than the test SAP, other service endpoints (For example: SAPs/SDP-Bindings) configured in the service can be over a LAG.

• The user needs to configure the resources of another port for use with the testhead OAM tool. The user can configure the resources of the internal virtual ports (if available) or configure the resources of a front-panel port for use with testhead OAM tool. Please see the CLI command config>system>loopback-no-svc-port in the 7210 SAS-Mxp, R6, R12, S, Sx, T Interface Configuration Guidefor more details. Interfaces to know if front-panel port resources are needed and use the command show>system>internal-loopback-ports [detail] to know if internal port resources are available in use by other applications. The port configured for testhead tool use cannot be shared with other applications that need the loopback port. The resources of the loopback port are used by the testhead tool for traffic generation.

  Note:

  On 7210 SAS-R6 and 7210 SAS-R12, the ports allocated for testhead OAM tool, the MAC swap OAM tool and the test SAP must be on the same line card and cannot be on different line cards.

• Port loopback with mac-swap is used at both ends and all services on the port on which the test SAP is configured and SAPs in the VPLS service, other than the test SAP should be shutdown or should not receive any traffic.

• The configured CIR/PIR rate is rounded off to the nearest available hardware rates. User is provided with an option to select the adaptation rule to use (similar to support available for QoS policies).

• The 7210 SAS supports all port speeds (that is, up to 1G rate). Users need to configure the appropriate loopback port to achieve a desired rate. For example, user must dedicate the resources of a 1G port, if intended to test rates to go up to 1G.

• ITU-T Y.1564 recommends to provide an option to configure the CIR step-size and the step-duration for the service configuration tests. This is not supported directly in 7210 SAS. It can be achieved by SAM or a third-party NMS system or an application with configuration of the desired rate and duration to correspond to the CIR step-size and step duration and repeating the test a second time, with a different value of the rate (that is, CIR step size) and duration (that is, step duration) and so on.

• Testhead waits for about 5 seconds at the end of the configured test duration before collecting statistics. This allows for all in-flight packets to be received by the node and accounted for in the test measurements. User cannot start another test during this period.

• When using testhead to test bandwidth available between SAPs configured in a VPLS service, operators must ensure that no other SAPs in the VPLS service are exchanging any traffic, particularly BUM traffic and unicast traffic destined for either the local test SAP or the remote SAP. BUM traffic eats into the network resources which is also used by testhead traffic.

• It is possible that test packets (both data and marker packets) remain in the loop created for testing when the tests are killed. This is highly probable when using QoS policies with very less shaper rates resulting in high latency for packets flowing through the network loop. User must remove the loop at both ends when the test is complete or when the test is stopped and wait for a suitable time before starting the next test for the same service, to ensure that packets drain out of the network for that service. If this is not done, then the subsequent tests may process and account these stale packets, resulting in incorrect results. Software cannot detect stale packets in the loop as it does not associate or check each and every packet with a test session

• Traffic received from the remote node and looped back into the test port (where the test SAP is configured) on the local end (that is, the end where the testhead tool is invoked) is dropped by hardware after processing (and is not sent back to the remote end). The SAP ingress QoS policies and SAP ingress filter policies must match the packet header fields specified by the user in the testhead profile, except that the source/destination MAC addresses are swapped.

• Latency is not be computed if marker packets are not received by the local node where the test is generated and printed as 0 (zero), in such cases. If jitter = 0 and latency > 0, it means that jitter calculated is less than the precision used for measurement. There is also a small chance that jitter was not actually calculated, that is, only one value of latency has been computed. This typically indicates a network issue, instead of a testhead issue.

• When the throughput is not met, FLR cannot be calculated. If the measured throughput is approximately +/-10% of the user configured rate, FLR value is displayed; else software prints ‟Not Applicable”. The percentage of variance of measured bandwidth depends on the packet size in use and the configured rate.

• Users must not use the CLI command to clear statistics of the test SAP port, testhead loopback port, or MAC swap loopback port when the testhead tool is running. The port statistics are used by the tool to determine the Tx/Rx frame count.

• Testhead tool generates traffic at a rate slightly above the CIR. The additional bandwidth is attributable to the marker packets used for latency measurements. This is not expected to affect the latency measurement or the test results in a significant way.

• If the operational throughput is 1kbps and it is achieved in the test loop, the throughput computed could still be printed as 0 if it is less than 1Kb/s (0.99 kb/s, for example). Under such cases, if FLR is PASS, the tool indicates that the throughput has been achieved.

• The testhead tool displays a failure result if the received count of frames is less than the injected count of frames, even though the FLR may be displayed as 0. This happens because of truncation of FLR results to 6 decimal places and can happen when the loss is very less.

• As the rate approaches 1Gbps or the maximum bandwidth achievable in the loop, user needs to account for the marker packet rate and the meter behavior while configuring the CIR rate. That is, if the user needs to test 1Gbps for 512 bytes frame size, then they will need to configure about 962396Kbps, instead of 962406Kbps, the maximum rate that can be achieved for this frame-size. In general, they would need to configure about 98%-99% (based on packet size) of the maximum possible rate to account for marker packets when they need to test at rates which are closer to bandwidth available in the network. The reason for this is that at the maximum rate, injection of marker packets by CPU will result in drops of either the injected data traffic or the marker packets themselves, as the net rate exceeds the capacity. These drops cause the testhead to always report a failure, unless the rate is marginally reduced.

• The testhead uses the Layer 2 rate, which is calculated by subtracting the Layer 1 overhead that is caused when the IFG and Preamble are added to every Ethernet frame (typically about 20 bytes (IFG = 12 bytes and Preamble = 8 bytes)). The testhead tool uses the user-configured frame size to compute the Layer 2 rate and does not allow the user to configure a value greater than that rate. For 512-byte Ethernet frames, the L2 rate is 962406 Kb/s and the Layer 1 rate is 1 Gb/s.

• It is not expected that the operator will use the testhead tool to measure the throughput or other performance parameters of the network during the course of network event. The network events could be affecting the other SAP/SDP-Binding/PW configured in the service. Examples are transition of a SAP because of G8032 ring failure, transition of active/ standby SDP-Binding/PW because of link or node failures.

• The 2-way delay (also known as ‟latency”) values measured by the testhead tool is more accurate than obtained using OAM tools, as the timestamps are generated in hardware.

• The 7210 SAS does not support color-aware metering on access SAP ingress, therefore, any color-aware packets generated by the testhead is ignored on access SAP ingress. 7210 SAS service access port, access-uplink port, or network port can mark the packets appropriately on egress to allow the subsequent nodes in the network to differentiate the in-profile and out-of-profile packets and provide them with appropriate QoS treatment. The 7210 SAS access-uplink ingress and network port ingress is capable of providing appropriate QoS treatment to in-profile and out-of-profile packets.

• The marker packets are sent over and above the configured CIR or PIR rate, the tool cannot determine the number of green packets injected and the number of yellow packets injected individually. Therefore, marker packets are not accounted in the injected or received green or in-profile and yellow or out-of-profile packet counts. They are only accounted for the Total Injected and the Total Received counts. So, the FLR metric accounts for marker packet loss (if any), while green or yellow FLR metric does not account for any marker packet loss.

• Marker packets are used to measure green or in-profile packets latency and jitter and the yellow or out-of-profile packets latency and jitter. These marker packets are identified as green or yellow based on the packet marking (Example: dot1p). The latency values can be different for green and yellow packets based on the treatment provided to the packets by the network QoS configuration.

  The following table describes SAP encapsulation supported by the testhead tool.

  Table 9. SAP encapsulations supported by testhead tool

  Epipe service configured with svc-sap-type	Test SAP encapsulations
  null-star	Null, :* , 0.* , Q.*
  Any	Null , :0 , :Q , :Q1.Q2
  dot1q-preserve	:Q

• The following combination of 1G and 10G port can be used, as long as the rate validated is less than or equal to 1Gb/s:

  • The test SAP is a 10G port, the uplink is 1G port and other ports (that is, uplink, MAC swap, and testhead) are 1G port.

  • The test SAP is a 10G port, the uplink is a 10G port and other ports (that is, MAC swap and testhead) are 1G port.

  • The test SAP is a 1G port, the uplink is a 10G port and other ports (that is, MAC swap and testhead) are 1G port.

This is the overall collection of different tests, the test parameters, measurement intervals, and mapping to configured storage models. It is the overall container that defines the attributes of the session:

• session type

  A session type is the impetus of the test, which is either proactive (default) or on-demand. Individual test timing parameters are influenced by this setting. A proactive session will start immediately following the execution of a no shutdown command for the test. A proactive test will continue to execute until a manual shutdown stops the individual test. On-demand tests will also start immediately following the no shutdown command. However, the operator can override the no test-duration default and configure a fixed amount of time that the test will execute, up to 24 hours (86400 seconds). If an on-demand test is configured with a test-duration, it is important to shut down tests when they are completed. In the event of a high availability event causing the backup CPM to become the active CPM, all on-demand tests that have a test-duration statement will restart and run for the configured amount of time regardless of their progress on the previously active CPM.

• test family

  The test family is the main branch of testing that addresses a specific technology. The available test for the session are based on the test family. The destination, source, and priority are common to all tests under the session and are defined separately from the individual test parameters.

• test parameters

  Test parameters are the parameters included in individual tests, as well as the associated parameters including start and stop times and the ability to activate and deactivate the individual test.

• measurement interval

  A measurement interval is the assignment of collection windows to the session with the appropriate configuration parameters and accounting policy for that specific session.

The session can be viewed as the single container that brings all aspects of individual tests and the various OAM-PM components under a single umbrella. If any aspects of the session are incomplete, the individual test cannot be activated with a no shutdown command, and an "Invalid Ethernet session parameters" error will occur.

A number of standards bodies define performance monitoring packets that can be sent from a source, processed, and responded to by a reflector. The protocols available to carry out the measurements are based on the test family type configured for the session.

Ethernet PM delay measurements are carried out using the Two Way Delay Measurement Protocol version 1 (DMMv1) defined in Y.1731 by the ITU-T. This allows for the collection of Frame Delay (FD), InterFrame Delay Variation (IFDV), Frame Delay Range (FDR), and Mean Frame Delay (MFD) measurements for round trip, forward, and backward directions.

DMMv1 adds the following to the original DMM definition:

• the Flag Field (1 bit – LSB) is defined as the Type (Proactive=1 | On-Demand=0)

• the TestID TLV (32 bits) is carried in the Optional TLV portion of the PDU

DMMv1 and DMM are backwards compatible and the interaction is defined in Y.1731 ITU-T-2011 Section 11 "OAM PDU validation and versioning".

Ethernet PM loss measurements are carried out using Synthetic Loss Measurement (SLM), which is defined in Y.1731 by the ITU-T. This allows for the calculation of Frame Loss Ratio (FLR) and availability.

A session can be configured with one or more tests. Depending on the session test type family, one or more test configurations may need to be included in the session to gather both delay and loss performance information. Each test that is configured shares the common session parameters and the common measurement intervals. However, each test can be configured with unique per-test parameters. Using Ethernet as an example, both DMM and SLM would be required to capture both delay and loss performance data.

Each test must be configured with a test ID as part of the test parameters, which uniquely identifies the test within the specific protocol. A test ID must be unique within the same test protocol. Again using Ethernet as an example, DMM and SLM tests within the same session can use the same test ID because they are different protocols. However, if a test ID is applied to a test protocol (like DMM or SLM) in any session, it cannot be used for the same protocol in any other session. When a test ID is carried in the protocol, as it is with DMM and SLM, this value does not have global significance. When a responding entity must index for the purpose of maintaining sequence numbers, as in the case of SLM, the test ID, Source MAC, and Destination MAC are used to maintain the uniqueness of the responder. This means that the test ID has only local, and not global, significance.

A measurement interval is a window of time that compartmentalizes the gathered measurements for an individual test that have occurred during that time. Allocation of measurement intervals, which equates to system memory, is based on the metrics being collected. This means that when both delay and loss metrics are being collected, they allocate their own set of measurement intervals. If the operator is executing multiple delay and loss tests under a single session, then multiple measurement intervals will be allocated, with one interval allocated per criteria per test.

Measurement intervals can be 15 minutes (15-min), one hour (1-hour) and 1 day (1-day) in duration. The boundary-type defines the start of the measurement interval and can be aligned to the local time of day clock, with or without an optional offset. The boundary-type can be aligned using the test-aligned option, which means that the start of the measurement interval coincides with the activation of the test. By default the start boundary is clock-aligned without an offset. When this configuration is deployed, the measurement interval will start at zero, in relation to the length. When a boundary is clock-aligned and an offset is configured, the specified amount of time will be applied to the measurement interval. Offsets are configured on a per-measurement interval basis and only applicable to clock-aligned measurement intervals. Only offsets less than the measurement interval duration are allowed. The following table describes some examples of the start times of each measurement interval.

Although test-aligned approaches may seem beneficial for simplicity, there are some drawbacks that need to be considered. The goal of the time-based and well defined collection windows allows for the comparison of measurements across common windows of time throughout the network and for relating different tests or sessions. It is suggested that proactive sessions use the default clock-aligned boundary type. On-demand sessions may make use of test-aligned boundaries. On-demand tests are typically used for troubleshooting or short term monitoring that does not require alignment or comparison to other PM data.

The statistical data collected and the computed results from each measurement interval are maintained in volatile system memory by default. The number of intervals stored is configurable per measurement interval. Different measurement intervals will have different defaults and ranges. The interval-stored parameter defines the number of completed individual test runs to store in volatile memory. There is an additional allocation to account for the active measurement interval. To look at the statistical information for the individual tests and a specific measurement interval stored in volatile memory, the show oam-pm statistics … interval-number command can be used. If there is an active test, it can be viewed by using the interval number 1. In this case, the first completed record would be interval number 2, and previously completed records would increment up to the maximum intervals stored value plus one.

As new tests for the measurement interval are completed, the older entries are renumbered to maintain their relative position to the current test. If the retained test data for a measurement interval consumes the final entry, any subsequent entries cause the removal of the oldest data.

There are drawbacks to this storage model. Any high availability function that causes an active CPM switch will flush the results that are in volatile memory. Another consideration is the large amount of system memory consumed using this type of model. With the risks and resource consumption this model incurs, an alternate method of storage is supported.

An accounting policy can be applied to each measurement interval to write the completed data in system memory to non-volatile flash memory in an XML format. The amount of system memory consumed by historically completed test data must be balanced with an appropriate accounting policy. Nokia recommends that only necessary data be stored in non-volatile memory to avoid unacceptable risk and unnecessary resource consumption. It is further suggested that a large overlap between the data written to flash memory and stored in volatile memory is unnecessary.

The statistical information in system memory is also available through SNMP. If this method is chosen, a balance must be struck between the intervals retained and the times at which the SNMP queries collect the data. Determining the collection times through SNMP must be done with caution. If a file is completed while another file is being retrieved through SNMP, then the indexing will change to maintain the relative position to the current run. Correct spacing of the collection is key to ensuring data integrity.

The OAM-PM XML file contains the keywords and MIB references described in the following table.

By default, the 15-min measurement interval stores 33 test runs (32+1) with a configurable range of 1 to 96, and the 1-hour measurement interval stores 9 test runs (8+1) with a configurable range of 1 to 24. The only storage for the 1-day measurement interval is 2 (1+1). This value for the 1-day measurement interval cannot be changed.

All three measurement intervals may be added to a single session if required. Each measurement interval that is included in a session is updated simultaneously for each test that is executing. If a measurement interval length is not required, it should not be configured. In addition to the three predetermined length measurement intervals, a fourth ‟always on” raw measurement interval is allocated at test creation. Data collection for the raw measurement interval commences immediately following the execution of a no shutdown command. It is a valuable tool for assisting in real-time troubleshooting as it maintains the same performance information and relates to the same bins as the fixed length collection windows. The operator may clear the contents of the raw measurement interval and flush stale statistical data to look at current conditions. This measurement interval has no configuration options, cannot be written to flash memory, and cannot be disabled; It is a single never-ending window.

Memory allocation for the measurement intervals is performed when the test is configured. Volatile memory is not flushed until the test is deleted from the configuration, a high availability event causes the backup CPM to become the newly active CPM, or some other event clears the active CPM system memory. Shutting down a test does not release the allocated memory for the test.

Measurement intervals also include a suspect flag. The suspect flag is used to indicate that data collected in the measurement interval may not be representative. The flag will be set to true only under the following conditions:

• Time of day clock is adjusted by more than 10 seconds.

• Test start does not align with the start boundary of the measurement interval. This would be common for the first execution for clock aligned tests.

• Test stopped before the end of the measurement interval boundary

The suspect flag is not set when there are times of service disruption, maintenance windows, discontinuity, low packet counts, or other such events. Higher level systems would be required to interpret and correlate those types of event for measurement intervals which executed during the time that relate to the specific interruption or condition. Because each measurement interval contains a start and stop time, the information is readily available for higher level systems to discount the specific windows of time.

There are two main metrics that are the focus of OAM-PM: delay and loss. The different metrics have two unique storage structures and will allocate their own measurement intervals for these structures. This occurs regardless of whether the performance data is gathered with a single packet or multiple packet types.

Delay metrics include Frame Delay (FD), InterFrame Delay Variation (IFDV), Frame Delay Range (FDR) and Mean Frame Delay (MFD). Unidirectional and round trip results are stored for each metric:

• Frame Delay

  The Frame Delay is the amount of time required to send and receive the packet.

• InterFrame Delay Variation

  IFDV is the difference in the delay metrics between two adjacent packets.

• Frame Delay Range

  The Frame Delay Range is the difference between the minimum frame delay and the individual packet

• Mean Frame Delay

  The Mean Frame Delay is the mathematical average for the frame delay over the entire window.

FD, IFDV and FDR statistics are binnable results. FD, IFDV, FDR and MFD all include minimum, maximum, and average values. Unidirectional and round trip results are stored for each metric.

Unidirectional frame delay and frame delay range measurements require exceptional time of day clock synchronization. If the time of day clock does not exhibit extremely tight synchronization, unidirectional measurements will not be representative. In one direction, the measurement will be artificially increased by the difference in the clocks. In the other direction, the measurement will be artificially decreased by the difference in the clocks. This level of clocking accuracy is not available with NTP. To achieve this level of time of day clock synchronization, Precision Time Protocol (PTP) 1588v2 should be considered.

Round trip metrics do not require clock synchronization between peers, since the four timestamps allow for accurate representation of the round trip delay. The mathematical computation removes remote processing and any difference in time of day clocking. Round trip measurements do require stable local time of day clocks.

Any delay metric that is negative will be treated as zero and placed in bin 0, the lowest bin which has a lower boundary of 0 microseconds.

Delay results are mapped to the measurement interval that is active when the result arrives back at the source.

There are no supported log events based on delay metrics.

Loss metrics are only unidirectional and will report frame loss ratio (FLR) and availability information. Frame loss ratio is the computation of loss (lost/sent) over time. Loss measurements during periods of unavailability are not included in the FLR calculation as they are counted against the unavailability metric.

Availability requires relating three different functions. First, the individual probes are marked as available or unavailable based on sequence numbers in the protocol. A number of probes are rolled up into a small measurement window, typically 1 s. Frame loss ratio is computed over all the probes in a small window. If the resulting percentage is higher than the configured threshold, the small window is marked as unavailable. If the resulting percentage is lower than the threshold, the small window is marked as available. A sliding window is defined as some number of small windows, typically 10. The sliding window is used to determine availability and unavailability events. Switching from one state to the other requires every small window in the sliding window to be the same state and different from the current state.

Availability and unavailability counters are incremented based on the number of small windows that have occurred in all available and unavailable windows.

Availability and unavailability using synthetic loss measurements is meant to capture the loss behavior for the service. It is not meant to capture and report on service outages or communication failures. Communication failures of a bidirectional or unidirectional nature must be captured using some other means of connectivity verification, alarming, or continuity checking. During times of complete or extended failure periods it becomes necessary to timeout individual test probes. It is not possible to determine the direction of the loss because no response packets are being received back on the source. In this case, the statistics calculation engine maintains the previous state, updating the appropriate directional availability or unavailability counter. At the same time, an additional per-direction undetermined counter is updated. This undetermined counter is used to indicate that the availability or unavailability statistics could not be determined for a number of small windows.

During connectivity outages, the higher level systems can be used to discount the loss measurement interval, which covers the same span as the outage.

Availability and unavailability computations may delay the completion of a measurement interval. The declaration of a state change or the delay to a closing a measurement interval could be equal to the length of the sliding window and the timeout of the last packet. Closing of a measurement interval cannot occur until the sliding window has determined availability or unavailability. If the availability state is changing and the determination is crossing two measurement intervals, the measurement interval will not complete until the declaration has occurred. Typically, standard bodies indicate the timeout per packet. In the case of Ethernet, DMMv1, and SLM, timeout values are set at 5 s and cannot be configured.

There are no log events based on availability or unavailability state changes.

During times of availability, there can be times of high loss intervals (HLI) or consecutive high loss intervals (CHLI). These are indicators that the service was available but individual small windows or consecutive small windows experienced frame loss ratios exceeding the configured acceptable limit. A HLI is any single small window that exceeds the configured frame loss ratio. This could equate to a severely errored second, assuming the small window is one second. A CHIL is a consecutive high loss interval that exceeds a consecutive threshold within the sliding window. Only one HLI will be counted for a window.

Availability can only be reasonably determined with synthetic packets. This is because the synthetic packet is the packet being counted and provides a uniform packet flow that can be used for the computation. Transmit and receive counter-based approaches cannot reliably be used to determine availability because there is no guarantee that service data is on the wire, or the service data on the wire uniformity could make it difficult to make a declaration valid.

The following figure shows loss in a single direction using synthetic packets, and demonstrates what happens when a possible unavailability event crosses a measurement interval boundary. In the diagram, the first 13 small windows are all marked available (1), which means that the loss probes that fit into each of those small windows did not equal or exceed a frame loss ratio of 50%. The next 11 small windows are marked as unavailable, which means that the loss probes that fit into each of those small windows were equal to or above a frame loss ratio of 50%. After the 10th consecutive small window of unavailability, the state transitions from available to unavailable. The 25th small window is the start of the new available state which is declared following the 10th consecutive available small window. Notice that the frame loss ratio is 00.00%; this is because all the small windows that are marked as unavailable are counted toward unavailability, and therefore are excluded from impacting the FLR. If there were any small windows of unavailability that were outside of an unavailability event, they would be marked as HLI or CHLI and be counted as part of the frame loss ratio.

Bin groups are templates that are referenced by the session. Three types of binnable statistics are available: FD, IFDV, and FDR, all of which are available in forward, backward, and round trip directions. Each of these metrics can have up to ten bin groups configured to group the results. Bin groups are configured by indicating a lower boundary. Bin 0 has a lower boundary that is always zero and is not configurable. The microsecond range of the bins is the difference between the adjacent lower boundaries. For example, bin-type fd bin 1 configured with lower-bound 1000 means that bin 0 will capture all frame delay statistics results between 0 and 1 ms. Bin 1 will capture all results above 1 ms and below the bin 2 lower boundary. The last bin to be configured would represent the bin that collects all the results at and above that value. Not all ten bins must be configured.

Each binnable delay metric type requires their own values for the bin groups. Each bin in a type is configurable for one value. It is not possible to configure a bin with different values for round trip, forward, and backward. Consideration must be given to the configuration of the boundaries that represent the important statistics for that specific service.

As stated earlier in this section, this is not a dynamic environment. If a bin group is being referenced by any active test the bin group cannot shutdown. To modify the bin group it must be shut down. If the configuration of a bin group must be changed, and a large number of sessions are referencing the bin group, migrating existing sessions to a new bin group with the new parameters can be considered to reduce the maintenance window. To modify any session parameter, every test in the session must be shut down.

Bin group 1 is the default bin group. Every session requires a bin group to be assigned. By default, bin group 1 is assigned to every OAM-PM session that does not have a bin group explicitly configured. Bin group 1 cannot be modified. The bin group 1 configuration parameters are as follows:

-------------------------------------------------------------------------------
Configured Lower Bounds for Delay Measurement (DMM) Tests, in microseconds
-------------------------------------------------------------------------------
Group Description                    Admin Bin     FD(us)    FDR(us)   IFDV(us)
-------------------------------------------------------------------------------
1     OAM PM default bin group (not*    Up   0          0          0          0
                                             1       5000       5000       5000
                                             2      10000          -          -
-------------------------------------------------------------------------------

The following figure shows the architecture of all of the OAM-PM concepts previously described. It shows a more detailed hierarchy than previously shown in the introduction. This shows the relationship between the tests, the measurement intervals, and the storage of the results.

The following configuration examples are used to demonstrate the different show and monitoring commands available to check OAM-PM.