CAUTION

Deployment failure

Check component hardware

Perform a file system check on each component by entering the following for each file system device as the root user on the component station:

# e2fsck -n file_system ↵

where file_system is a specification such as /dev/sdan for a physical disk, or /dev/vdan for a virtual disk

If the check passes, the output is similar to the following

/dev/sdan: clean, a/b files, x/y blocks

If the check indicates a failure, you must correct the disk corruption before you continue.

Check each component station for system error messages; enter the following as the root user on the station:

# grep -i error /var/log/messages ↵

If no error messages are present, the command returns nothing. Otherwise, investigate and resolve the issues indicated by the error messages that are displayed.

Perform “How do I test NFM-P disk performance?” in the NSP System Administrator Guide on each component station to ensure that the disk speed and latency meet or exceed your system requirements.

Check component OS configuration

Verify that the /etc/hosts file contains the required host entry for each NFM-P component; enter the following as the root user on each station, and ensure that each required entry is present and correctly specified, as described in Network requirements:

# cat /etc/hosts ↵

Verify that the /etc/nsswitch.conf file is configured correctly; enter the following as the root user on each component station:

# cat /etc/nsswitch.conf ↵

Ensure that “files” is the first entry for passwd, shadow, group, and hosts, as shown in the following:

passwd:     files nis

shadow:     files nis

group:      files nis

hosts:      files dns myhostname

Verify that the OS version is compatible with the NFM-P release, as described in the NSP Planning Guide; enter the following as the root user on each component station:

# cat /etc/redhat-release ↵

Red Hat Enterprise Linux Server release R.r (codename)

Verify that the disk partitions are correctly allocated and sized; enter the following commands as the root user on each component station, and check the output against the specifications in Chapter 2, NSP disk setup and partitioning:

# lsblk ↵

# df -PH ↵

Verify that the required RHEL OS packages are installed; enter the following as the root user on each component station; see Chapter 3, RHEL OS deployment for the NSP for information about the required packages:

# dnf list installed ↵

Verify NE resynchronization status

Use the Discovery Manager in the NFM-P GUI to ensure that no NEs are undergoing or pending resynchronization.

1. Open an NFM-P GUI client.

2. Choose Administration→Discovery Manager from the NFM-P main menu. The Discovery Manager (Edit) form opens.

3. Click on the Resync Status tab.

4. Ensure that no NEs are in either of the following states:

   • In Progress

   • Requested

5. Close the Discovery Manager form.

Verify NFM-P license

Verify that you have a valid license; perform the following steps on each main server station.

Note: Ensure that you do not modify the contents of a license file; otherwise, the license is unusable.

1. Enter the following as the root user:

   # cat /sys/devices/virtual/dmi/id/product_uuid ↵

   The station UUID is displayed.

2. Record the UUID.

3. Unzip the license file; enter the following:

   # tar -xvf license_file ↵

   where license_file is the path of the compressed license file

   The nfmpLicense.xml file is extracted to the current directory.

4. Open the nfmpLicense.xml file for viewing.

5. Review the license parameters to ensure the following:

   • The UUIDs match the station UUIDs; if the same license is used for the two main servers in a redundant deployment, the license includes both UUIDs.

   • The license is for the correct release. A license is specific to a major release only; a new license is not required for an upgrade to a different minor release in the same major release.

   • The license enables the required functions.

   • The licensed capacities are correct and sufficient for you network.

Back up NFM-P configuration

Copy the following configuration files to a secure location on a station that is not affected by the upgrade:

• from each main server station:

  • contents of /opt/nsp/nfmp/server/nms/config

  • /opt/nsp/nfmp/config/nms/config/main_config.xml

• from each auxiliary server station:

  • contents of /opt/nsp/nfmp/auxserver/nms/config

  • /opt/nsp/nfmp/config/nms/config/aux_config.xml

• from each main database station:

  • /opt/nsp/nfmp/samdb/install/config/dbconfig.properties

  • /opt/nsp/nfmp/config/nms/config/db_config.xml

Remove outdated logs

An NFM-P main server may retain logs saved during a previous upgrade in the following directory:

/opt/nsp/nfmp/server/nms/log/previous_log

The files in the directory may consume excessive disk space. If the directory exists on a main server, it is strongly recommended that you remove the files from the directory before the upgrade.

1. Move the files to a secure archive location on a station that does not host an NSP component.

2. Delete the files.

Check and configure firewalls

You must ensure that each firewall between the system components allows the required traffic to pass between the components, or is disabled. You can configure and enable the firewalls after the upgrade, if required.

1. Ensure that each firewall allows the required traffic to pass. See the NSP Planning Guide for a list of the ports that must be open on each component.

   Note: The RHEL firewalld service must be configured using the firewalld rules in the NSP Planning Guide, which describes using NFM-P templates for rule creation.

2. Disable each firewall; see the external firewall documentation, or perform To disable the RHEL firewalld service.

Download installation files

Download the following NFM-P installation files to an empty directory on a station that is not affected by the upgrade activity:

Note: The station must be reachable by each station that is to host an NFM-P main server or main database.

• nsp-nfmp-jre-R.r.p-rel.v.rpm

• nsp-nfmp-config-R.r.p-rel.v.rpm

• nsp-nfmp-nspos-R.r.p-rel.v.rpm

• nsp-nfmp-main-server-R.r.p-rel.v.rpm

• nsp-nfmp-oracle-R.r.p-rel.v.rpm

• nsp-nfmp-main-db-R.r.p-rel.v.rpm

• nsp-nfmp-nodeexporter-R.r.p-rel.v.rpm, if the NFM-P is in a shared-mode deployment and you want to forward NFM-P system metrics to the NSP

• OracleSw_PreInstall.sh

where

R.r.p is the NSP release identifier, in the form MAJOR.minor.patch

v is a version identifier

Validate database

Before you upgrade a main database, you must ensure that the main database contains only valid records, or the upgrade fails.

Note: In a redundant system, you must perform the validation on the primary main database station.

Log in as the root user on the main database station.

Transfer the following downloaded file to an empty directory on the main database station:

• OracleSw_PreInstall.sh

Navigate to the directory that contains the OracleSw_PreInstall.sh file.

Enter the following:

# chmod +x OracleSw_PreInstall.sh ↵

Perform the following steps to validate the Oracle database and resolve any conflicts that may prevent an upgrade.

Note: If the validation check reports a small number of errors, for example, a few duplicate and invalid instances of an object, deleting the invalid instances manually may be sufficient. A large number of errors may indicate a significant problem that requires expert attention; in such a case, contact technical support for assistance.

1. Enter the following:

   # ./OracleSw_PreInstall.sh -check ↵

   The following prompt is displayed:

   Enter the password for the "SYS" Oracle user (terminal echo is off):

2. Enter the SYS user password.

   The following messages are displayed:

   Logging Oracle pre install checks to log_file

   In upgrade check mode, this script does not modify the system.

   About to validate that the database can be upgraded to release.

   Found the NFM-P main database installation directory /opt/nsp/nfmp/db/install.

   If the validation is successful, the following messages and prompt are displayed:

   INFO: Database upgrade validation passed.

3. If the validation is successful, go to Step 20.

4. If the script detects one or more invalid items, for example, an NE at a release that the new NFM-P software does not support, an incomplete deployment, or other upgrade restriction, one line like the following is displayed for each item:

   ERROR: Error message

   The following is displayed as the script exits.

   ERROR: The database cannot be upgraded. Please fix the above errors and re-run this script.

   Remove the upgrade restriction. For example, clear an incomplete deployment, or upgrade an unsupported NE to a release that the new software supports.

5. Run the script again; go to substep 1.

Verify database archive log synchronization

If the system is redundant, ensure that no archive log gap exists between the primary and standby main databases.

Note: If you attempt a database upgrade when an archive log gap exists, the upgrade fails.

1. Open an NFM-P GUI client.

2. View the Standby DB entry in the GUI status bar.

3. If the entry reads “Database archive log gap”, you must reinstantiate the standby database. Otherwise, go to Step 21.

4. Choose Administration→System Information from the main menu. The System Information form opens.

5. Click Re-Instantiate Standby.

6. Click Yes to confirm the action. The reinstantiation begins, and the GUI status bar displays reinstantiation information.

   Note: Database reinstantiation takes considerable time if the database contains a large amount of statistics data.

   You can also use the System Information form to monitor the reinstantiation progress. The Last Attempted Standby Re-instantiation Time is the start time; the Standby Re-instantiation State changes from In Progress to Success when the reinstantiation is complete.

7. When the reinstantiation is complete, close the System Information form.

Back up database

Open an NFM-P GUI client.

Choose Administration→Database from the main menu. The Database Manager form opens.

Click on the Backup tab.

CAUTION Service Disruption

The disk partition that is to contain the database backup must have sufficient space for the database backup file set.

Ensure that the backup directory is at least five times as large as the expected database backup size. For more information, contact technical support or see the NSP Planning Guide.

CAUTION Data Loss

Before the NFM-P performs a database backup, it deletes the contents of the specified backup directory.

Ensure that the backup directory that you specify does not contain files that you need to retain.

CAUTION Data Loss

The backup directory that you specify must not include the main database installation directory, or data loss may occur.

Ensure that the directory path does not include /opt/nsp/nfmp/db.

Note: The backup directory that you specify must be a directory on a local mounted partition.

Note: The Oracle management user requires read and write permissions on the backup directory.

Note: If the NFM-P system is independent, rather than part of a shared-mode NSP deployment, a main database backup performed using the NFM-P client GUI also backs up the local Neo4j and PostgreSQL databases; a backup performed from a CLI does not. The Neo4j and PostgreSQL backup files may be required in the event that the upgrade fails and is to be rolled back.

The Neo4j and PostgreSQL backup files are stored on the standalone or primary main server in the /opt/nsp/os/backup directory.

Configure the following parameters:

• Manual Backup Directory

• Enable Backup File Compression

Click Apply.

Click Full Backup.

Click OK. The database backup begins, and the Backup State indicator reads In Progress.

Note: Depending on the database size, a backup may take considerable time.

Monitor the Backup State indicator, which is dynamically updated. The indicator displays Success when the backup is complete.

When the backup is complete, close the Database Manager (Edit) form.

Transfer the following backup file sets to a secure location on a separate station that is unaffected by the upgrade activity:

• Oracle database—copy from the specified backup location in Step 24 on the standalone or primary main database station

• Neo4j and PostgreSQL databases—copy from the /opt/nsp/os/backup directory on the standalone or primary main server station

Back up custom configuration files

CAUTION Service Disruption

An NFM-P upgrade does not preserve all non-default settings in configuration files such as nms-server.xml.

If an NFM-P configuration file contains non-default settings that you want to retain after an upgrade, contact technical support for assistance before the upgrade.

Note: At the beginning of an NFM-P main or auxiliary server upgrade, specific configuration and log files are copied to a directory under the installation directory; the directory name includes a timestamp. The directories below the main server installation directory are then deleted. If you have created or customized a file below the main server installation directory, you risk losing the file unless you create a backup copy.

Make a backup copy of each file that you have created or customized in or below the /opt/nsp/nfmp/server directory on each main server station, and store the backup files on a separate station that is not affected by the NFM-P upgrade activity.

Verify compatibility with external systems

Ensure that the new NFM-P software is compatible with the software release of each external system that connects to the NFM-P. Contact technical support for information about external system compatibility.

Close LogViewer

Close the LogViewer utility, if it is open.

Validate main server and GUI client firewall configuration

Confirm that the firewalls between the main servers and the single-user GUI clients and client delegate servers allow traffic to the HTTP or HTTPS port required for client access. Otherwise, you cannot install or upgrade a single-user client or client delegate server.

See the NSP Planning Guide for NFM-P port assignment information.

Verify NFM-P compatibility with managed NEs

You must confirm that the new NFM-P release supports the software release of each managed NE and pre-provisioned NE, as stated in the NSP NFM-P Network Element Compatibility Guide.

Note: See also NFM-P deployment requirements for additional important device-specific compatibility requirements.

Note: If the system that you are upgrading manages an NE as a GNE, and the new NFM-P release supports native management of the device type and release, you must unmanage and delete the GNE before you attempt the upgrade. After the upgrade, the NFM-P can discover and manage the device as a native NE instead of a GNE.

Perform one of the following for each managed NE at an unsupported release.

1. Upgrade the device software to a release that the new NFM-P software supports; see the appropriate device documentation and the NSP NFM-P User Guide for information about performing NE software upgrades.

2. Remove the NE from the NFM-P managed network, as described in the NSP NFM-P User Guide.

   1. Unmanage the NE.

   2. Delete the NE from the managed network.

   3. Administratively disable or remove the discovery rule element for the NE.

3. If the NE is a pre-provisioned NE, delete the pre-provisioned NE using the NFM-P Pre-Provisioned NE Manager.

Clear CPAM checkpoints

An NFM-P main server upgrade requires additional time if CPAM checkpoints are retained. The additional time varies, depending on the platform resources, managed network size, and checkpoint schedule. To reduce the upgrade time, remove the CPAM checkpoints, as described in the NSP NFM-P Control Plane Assurance Manager User Guide.

Gather required information

Choose Administration→System Information from the main menu. The System Information form opens.

Record the following information:

• Domain Name

• Primary Server panel:

  • IP Address

  • Host Name

  • Status

• Primary Database Server panel:

  • Database Name

  • Instance Name

  • IP Address

  • Host Name

If the system is redundant, record the following additional information:

• Standby Server panel:

  • IP Address

  • Host Name

  • Status

• Standby Database Server panel:

  • Database Name

  • Instance Name

  • IP Address

  • Host Name

If the system includes one or more auxiliary servers, click on the Auxiliary Servers tab; otherwise, go to Step 44.

A list of auxiliary servers is displayed.

Perform the following steps for each auxiliary server listed on the form.

1. Select the auxiliary server and click Properties. The Auxiliary Server [Edit] form opens.

2. Record the following information for use during the upgrade:

   • Host Name

   • Auxiliary Server Type

   • Server Status

   • Public IP address

   • Private IP address, if displayed

3. Close the Auxiliary Server [Edit] form.

Click on the Auxiliary Services tab. Each Preferred auxiliary server entry has a check mark in the Selected column.

Record the hostname or IP address of each Preferred auxiliary server.

Note: The auxiliary servers are collectively referred to as the [Aux1] auxiliary servers In To upgrade a redundant Release 22.9 or later NFM-P system. Any other listed auxiliary servers are the Reserved auxiliary servers, and are collectively referred to as [Aux2] in the procedure.

If the system includes one or more client delegate servers, click on the Client Delegate Servers tab. Otherwise, go to Step 46.

Perform the following steps for each client delegate server listed on the form:

1. Select the client delegate server and click Properties. The client delegate server properties form opens.

2. Record the IP Address value for use during the upgrade.

3. Close the properties form.

Close the System Information form.

Obtain and record the following additional information for each main server:

• root user password

• nsp user password

• additional IP addresses, if NAT or multiple interfaces are used:

  • IP address that each main database must use to reach the main server

  • IP address that the GUI and XML API clients must use to reach the main server; the public IP address, if NAT is used

  • IP address that the auxiliary servers must use to reach the main server

  • private IP address, if NAT is used

Obtain and record the following additional main database information:

• root user password

• Oracle management user information:

  • username; installation default is oracle

  • password

  • group name; installation default is dba

• Oracle database user information:

  • username; installation default is samuser

  • password

• Oracle SYS user password

• additional database IP addresses, if NAT or multiple interfaces are used:

  • IP address that each main server must use to reach the database

  • IP address that each auxiliary server must use to reach the database

Close client sessions

Close the open GUI and XML API client sessions, as required.

1. Open a GUI client using an account with security management privileges, such as admin.

2. Choose Administration→Security→NFM-P User Security from the main menu. The NFM-P User Security - Security Management (Edit) form opens.

3. Click on the Sessions tab.

4. Click Search. The form lists the open GUI and XML API client sessions.

5. Identify the GUI session that you are using based on the value in the Client IP column.

6. Select all sessions except for the following:

   • the session that you are using

   • the sessions required to monitor the network during a redundant system upgrade

7. Click Close Session.

8. Click Yes to confirm the action.

9. Click Search to refresh the list and verify that only the required sessions are open.

10. Close the NFM-P User Security - Security Management (Edit) form.

Uninstall Mac OS X clients

Uninstall each single-user client installed on Mac OS X.

Note: You must use the uninstallation procedure in the documentation for the installed client release, and not the uninstallation procedure in this guide.

Close GUI client

If the GUI client that you are using is not required for network monitoring during the upgrade, close the client.

End of steps