STM concepts and components

OAM diagnostic tests

The NFM-P Service Test Manager (STM) provides access to a set of configurable in-band or out-of-band, packet-based OAM diagnostic tests that allow you to perform on-demand or scheduled verifications of SLA compliance and for network troubleshooting.

You can perform the following tasks with the STM:

• Create on-demand OAM diagnostic tests to provide proactive detection of service degradation and SLA verification and to verify end-to-end service performance. See Chapter 90, OAM diagnostic testsfor information about each OAM diagnostic test that can be configured with the NFM-P STM.

• Run discrete OAM diagnostic tests on concurrent groups of objects.

• Group OAM tests in an integrated STM test suite for concurrent execution to provide continual performance feedback such as latency, delay, packet loss, and threshold-crossing alerts. The test results are logged for monitoring and trend analysis.

• STM test suites can be immediately processed, scheduled for the automatic execution of tasks using NFM-P-based schedules at designated times, or retained for future use. An NFM-P schedule is configurable for one-time or ongoing task execution.

• Configure threshold-crossing parameters to generate alarms when rising or falling threshold values are reached due to the reach, latency, or jitter issues discovered by the OAM diagnostic tests.

• Automatically generate OAM tests based on object or topology changes.

Note: You can customize some STM-specific user and system preferences to change the default STM settings to meet your operational requirements. See the workflow in STM workflow for additional information.

Note: The NFM-P STM allows the deployment of the maximum number of tests to a 7450 ESS, 7750 SR, or 7950 XRS. The NFM-P raises an alarm when the number of tests on an NE is 60% of the configured maximum. Attempts to create or execute a test using the NFM-P fail when the number of deployed tests on an NE is too high.

Note: OAM tests on 7705 SAR-Hm NEs will not execute if the packet size is greater than 1600 octets.

STM test policies

To enable the automatic generation of tests within an STM test suite, the NFM-P requires that the STM test policy contains a set of test definitions that defines the pre- and post-processing rules. An STM test policy also specifies the order of execution for the generated tests. An STM test policy is applied to an STM test suite during test suite creation. See To configure an STM test policy for information about how to create an STM test policy.

You can re-apply an STM test policy under the following conditions:

• by clicking Generate on the Test Policy (Create|Edit) form when any test entities have changed

• by clicking Update Test Suites on the Test Policy (Create|Edit) form

An STM test policy can be used by multiple STM test suites but an STM test suite can have only one associated STM test policy.

Note: You can apply changes made to an STM test policy to all referencing STM test suites by clicking Update Test Suites on the Test Policy (Create|Edit) form.

STM test policy parameters can be configured to only display test results if a test fails or generates a threshold-crossing alarm. In large networks, this can substantially reduce the amount of test data that the NFM-P needs to collect.

An STM test policy is specific to one type of entity; for example, a VLL service or service tunnel. The test definitions in the policy are restricted to the tests that apply to the entity type specified in the policy.

STM test suites

The grouping of OAM diagnostic tests into an STM test suite allows an NFM-P operator to use one schedule for the periodic execution of multiple tests against multiple network objects; for example, services, NEs, or transport components.

An operator can choose to include existing tests, use the NFM-P to generate the OAM diagnostic tests that comprise an STM test suite, or both. Groups of tests in a suite can be configured to execute sequentially or concurrently. In addition, you can configure an STM test suite as an OAM validator to verify the operational status of a service. This can also be done on a one-time basis using the one-time service validation test; see OAM diagnostic tests for more information.

A test suite contains three test groups:

• First-run tests

  First-run tests are the tests in a suite that the NFM-P executes before the tests in the other groups. First-run tests are chosen from a list of existing tests and might typically include high-level diagnostics; for example, a service site ping or VPRN ping. No restrictions apply to the types of tests that are selectable as first-run tests.

• Generated tests

  Generated tests are created by the NFM-P for use against a specific network entity, based on the entity type specified in the suite and the specific tested entities that are named in the associated test policy. For example, a service site ping test policy associated with a three-site VPRN test suite causes the NFM-P to generate six tests: one site ping test from each site in the VPRN to the other two sites. When you change the configuration of a network entity, such as a service, you must regenerate the generated tests that apply to the entity. Test regeneration removes previously generated tests from a test suite.

• Last-run tests

  Last-run tests are the tests in a suite that the NFM-P executes after the tests in the other groups. Last-run tests are chosen from a list of existing tests and might typically include transport-layer diagnostics; for example, an LSP trace or a tunnel ping. No restrictions apply to the types of tests that are selectable as last-run tests.

Note: First-run and last-run tests do not apply to STM test suites where the MEF35 Mode parameter is enabled. See To create an STM test suite for more information.

To create an STM test suite that contains tests for different entity types, you can specify that the test suite applies to no specific entity type. In this way, you can create a group of disparate tests to which no test policy restrictions apply. Specifying None as the entity type in a test suite has the following effects:

• It allows you to choose any predefined test as a first-run or last-run test.

• It disables test generation in the test suite, because test generation requires a test policy that is based on a specific entity type.

Note: The NFM-P does not attempt to discover tests or test suites that are configured locally on an NE, for example, using a CLI.

Note: By default, the NFM-P suppresses alarms for suspended NEs. See Suspending device management for more information. If you attempt to modify an OAM test or test suite that is deployed to a suspended NE, the NFM-P raises an alarm.

To manage the system resources that test execution consumes, the NFM-P assigns a weight value to a test. When the NFM-P executes a test, it attempts to reserve the test weight from a resource pool, performs the test, then returns the test weight to the pool. The weight of a test suite is the sum of the weights of the individual tests in the suite. The NFM-P attempts to reserve the weight of the whole suite for the duration of suite execution. If the required weight for a test or test suite is unavailable, execution is halted and the Status value contained in the test result is set to Not Enough Resources.

You can create an OAM service validation test to verify the operational status of a service. The operational status of a service depends on the operational states of its service sites or instances. It is possible for a service to be operationally up when communication between sites is not operational. For example, a VRF can be operationally up but the routes to its peers might not be populated because of the routing policy, route target, or ACL configuration. The State Cause of the service indicates the success or failure of the OAM validation test, and therefore, service connectivity.

You can configure an OAM validator when you create a test suite and run the OAM validation test from the service configuration form.

Note: OAM validation tests are not supported for HVPLS.

STM test suite design considerations

Consider the following when you create, schedule, or run STM test suites.

• General design considerations:

  • A test, whether pre-existing or generated, is associated with only one STM test suite.

  • You can execute a generated test on demand, not just in the context of a test suite.

  • An NFM-P user who is assigned the admin or QoS/ACL management scope of command role can create and modify all tests, test policies, and test suites. A user who is assigned the service management scope of command role can create and modify only STM components that are related to services. A user who is assigned the topology management role can create and modify only STM components that are related to network transport elements.

• STM Test Suite (Create) form parameter considerations

  • OAM test suites in which the Validation Test Suite parameter is enabled are used to test the operational status of a service or service-related entity such as a service tunnel. The result of this validator test is indicated by the OAM Validation Failed state cause indicator on the General tab of the management form for the object.

  • The Entity Type parameter specifies that only test policies created for the same entity type are available for the test suite. The parameter also restricts the predefined tests that are available as first-run or last-run tests to those that apply to the entity type.

  • To create a test suite that includes tests for different entity types, specify None as the Entity Type parameter value. This allows you to choose any predefined test as a first-run or last-run test.

  • Specifying None as the Entity Type parameter value disables the generation of tests in a test suite; the policy the STM uses to generate tests must be associated with a specific entity type.

  • The Validation Test Suite parameter specifies that the test suite is used to validate the connectivity of the tested service entity to which it is applied. This is also referred to as a validator test suite. OAM validation tests are not supported for HVPLS. See To run a one-time validation test on a service for information about how to run a one-time validation test on a service. See To run an OAM validation test on a service tunnel for information about how to run a one-time validation test on a service tunnel.

  • If you are creating a test suite that contains an Ethernet CFM test definition, you must enable the NE Schedulable parameter.

• Scheduling and execution considerations:

  • The execution of STM test suites can be configured as scheduled tasks. See Chapter 5, NFM-P-based schedules for additional design consideration information about using the NFM-P to schedule STM test suites.

  • Using an NE schedule does not ensure that the target NEs perform the actions in an associated STM scheduled task in the order specified; the NEs that execute an NE scheduled task operate independently and are not directed by the NFM-P. As soon as an NE completes an action in an NE scheduled task, it performs the next action.

  • First-run and last-run tests that have the Continuously Executed and Accounting Files options enabled can be added to test suites.

  • When issuing an Execute or Stop Execution command for a test suite that has the Continuously Executed and Accounting Files options enabled, the execution or stopping of the first-run and last-run tests will be triggered.

  • You must click Validate on the associated service or service tunnel configuration form to run the OAM validator test suite. The result of this OAM validation is indicated by the OAM Validation Failed operational flag. See To run a one-time validation test on a service for information about how to run a one-time validation test on a service. See To run an OAM validation test on a service tunnel for information about how to run a one-time validation test on a service tunnel.

• Test result considerations:

  • You can view test suite results for an object from the Tests tab of the configuration form for the object.

  • The NFM-P database logs test results only for scheduled test suites that are associated with NFM-P-defined schedules. Logging for test suites that are associated with NE schedules is performed locally on each target NE.

  • Size constraint policies control the number of historical records that the NFM-P retains in the database. You can configure size constraint policies to limit the number of database objects that are associated with specific test types. See the NSP System Administrator Guide  for information about size constraint policies.