Contents


Troubleshooting Guide

Legal notice

Contents

About this document

Part I. Troubleshooting overview

1. NSP troubleshooting overview

Overview

The troubleshooting process

NSP and NFM-P troubleshooting tools

Process to troubleshoot a problem in the NSP

2. Obtaining Technical Assistance

Before you call support

Part II. Troubleshooting the system

3. Troubleshooting the NSP platform

To collect NSP log files

To retrieve a list of pods

To retrieve pod information

To recover pods

To recover executor pods

To retrieve a list of cluster members

To retrieve cluster member information

To retrieve detailed information about MDM servers

To rebalance NE load on MDM servers

To verify disk performance for etcd

To verify disk performance for NSP

Problem: NSP data synchronization is not 100%

Problem: Alarms not appearing for rapidly reoccurring faults

4. Troubleshooting the NFM-P platform

Troubleshooting the NFM-P

To collect NFM-P log files

Problem: Poor performance on a RHEL station

Problem: Device discovery fails because of exceeded ARP cache

Troubleshooting using the LogViewer

LogViewer overview

LogViewer GUI and Quick Links panel

LogViewer CLI

To display logs using the LogViewer GUI

To configure the LogViewer using the GUI

To search log files in a path

To show or hide buttons from the LogViewer main tool bar

To set highlight colors and fonts for LogViewer components and levels

To automatically show or hide log messages

To manage filters using the GUI Filter Manager

To specify a plug-in using the LogViewer GUI

To display logs using the LogViewer CLI

To configure the LogViewer CLI

To specify plug-ins using the CLI

Troubleshooting the NFM-P database

Database troubleshooting overview

Problem: NFM-P database corruption or failure

Problem: The database is running out of disk space

Problem: Frequent database backups create performance issues

Problem: An NFM-P database restore fails and generates a No backup sets error

Problem: NFM-P database redundancy failure

Problem: Primary or standby NFM-P database is down

Problem: Need to verify that Oracle database and listener services are started

Problem: Need to determine status or version of NFM-P database or Oracle proxy

Troubleshooting NFM-P server issues

NFM-P server troubleshooting overview

Problem: Cannot start an NFM-P server, or unsure of NFM-P server status

Problem: NFM-P server and database not communicating

Problem: An NFM-P server starts up, and then quickly shuts down

Problem: Client not receiving server heartbeat messages

Problem: Main server unreachable from RHEL client station

Problem: Excessive NFM-P server-to-client response time

Problem: Unable to receive alarms on the NFM-P, or alarm performance is degraded

Problem: All SNMP traps from managed devices are arriving at one NFM-P server, or no SNMP traps are arriving

Cannot manage new devices

Problem: Cannot discover more than one device, or device resynchronization fails

Problem: Slow or failed resynchronization with network devices

Problem: Statistics are rolling over too quickly

Problem: Y.1564 service test results not published to Kafka

Troubleshooting NFM-P clients

Problem: Cannot start NFM-P client, or error message during client startup

Problem: NFM-P client unable to communicate with NFM-P server

Problem: Delayed server response to client activity

Problem: Cannot place newly discovered device in managed state

Problem: User performs action, such as saving a configuration, but cannot see any results

Problem: Device configuration backup not occurring

Problem: NFM-P client GUI shuts down regularly

Problem: Configuration change not displayed on NFM-P client GUI

Problem: List or search function takes too long to complete

Problem: Cannot select some menu options or save some configurations

Problem: The NFM-P client GUI does not display NE user accounts created, modified, or deleted using the CLI

Part III. Troubleshooting the network

5. Network troubleshooting using NSP functions

Troubleshooting using NSP assurance functions

Troubleshooting services and connectivity

Onboarding an NE into NSP

Onboarding a service into NSP

LSP Throughput with Forecast reporting scenario

SAP Throughput reporting scenario

End-to-end NE troubleshooting scenario

End-to-end service troubleshooting scenario

End-to-end link troubleshooting scenario

End-to-end port troubleshooting scenario

Troubleshooting using Analytics

Analytics troubleshooting overview

Troubleshooting data collection

Troubleshooting data storage

Troubleshooting Analytics reporting

Troubleshooting using NSP workflows

Evaluating failed or slow workflow executions

6. Network troubleshooting using NFM-P

Troubleshooting services and connectivity

Service and connectivity diagnostics

Workflow to troubleshoot a service or connectivity problem

To identify whether a VPLS is part of an H-VPLS

To verify the operational and administrative states of service components

To verify the FIB configuration

To verify connectivity for all egress points in a service using MAC Ping and MAC Trace

To verify connectivity for all egress points in a service using MEF MAC Ping

To measure frame transmission size on a service using MTU Ping

To verify the end-to-end connectivity of a service using Service Site Ping

To verify the end-to-end connectivity of a service tunnel using Tunnel Ping

To verify end-to-end connectivity of an MPLS LSP using LSP Ping

To review the route for an MPLS LSP using LSP Trace

To review ACL filter properties

To view anti-spoof filters

To retrieve MIB information from a GNE using the snmpDump utility

Troubleshooting using the NE resync audit function

NE resync auditing overview

Workflow for NE resync auditing

To clear a Frame Size Problem (MTU Mismatch) alarm

To perform an NE resync audit

To view NE resync audit results using the NE audit manager

Troubleshooting network management LAN issues

Problem: All network management domain stations experience performance degradation

Problem: Lost connectivity to one or more network management domain stations

Problem: Another station can be pinged, but some functions are unavailable

Problem: Packet size and fragmentation issues

Troubleshooting using NFM-P client GUI warning messages

Client GUI warning message overview

To respond to a GUI warning message

Troubleshooting with Problems Encountered forms

Overview

To view additional problem information

To collect problem information for technical support

Troubleshooting using the NFM-P user activity log

User activity log overview

To identify the user activity for a network object

To identify the user activity for an NFM-P object

To navigate to the object of a user action

To view the user activity records of an object

To view the user activity performed during a user session