To upgrade a redundant Release 22.9 or later NFM-P system

Description

The following steps describe how to upgrade a collocated or distributed Release 22.9 or later main database and main server in a redundant deployment. The steps include links to procedures for installing and upgrading optional NFM-P components.

Ensure that you record the information that you specify, for example, directory names, passwords, and IP addresses.

Note: You require the following user privileges:

on each server station in the system — root, nsp
on each main database station — root

Note: The following RHEL CLI prompts in command lines denote the active user, and are not to be included in typed commands:

# —root user
bash$ —nsp user

Steps

Check pre-upgrade disk space

As part of the trial upgrade on a lab system in advance of a live upgrade, you must ensure that the available disk capacity on each NFM-P component remains within tolerance.

Note: If the disk usage on an NFM-P partition approaches or exceeds 80% after the trial upgrade, you may need to add disk capacity before you attempt the upgrade on a live system.

Perform the following steps on each of the following stations:

main server
auxiliary server
main database

Log in to the station as the root user.
Open a console window.
Enter the following:

# df -kh ↵

The usage information for each partition is displayed.
Record the information for each NFM-P partition; see the tables in Chapter 2, NSP disk setup and partitioning for the partition names and required capacities.

Stop and disable standby main server [Main2]

Open a GUI client to monitor the network during the upgrade.

Stop the standby main server.

Log in to the standby main server station as the nsp user.
Open a console window.
Enter the following:

bash$ cd /opt/nsp/nfmp/server/nms/bin ↵
Enter the following:

bash$ ./nmsserver.bash stop ↵
Enter the following:

bash$ ./nmsserver.bash appserver_status ↵

The server status is displayed; the server is fully stopped if the status is the following:

Application Server is stopped

If the server is not fully stopped, wait five minutes and then repeat this step. Do not perform the next step until the server is fully stopped.
Enter the following to switch to the root user:

bash$ su ↵

Disable the automatic main server startup so that the main server does not start in the event of a power disruption during the upgrade.

Enter the following:

# systemctl disable nspos-nspd.service ↵
Enter the following:

# systemctl disable nfmp-main-config.service ↵
Enter the following:

# systemctl disable nfmp-main.service ↵

Stop auxiliary servers [Aux2]

If the NFM-P system includes auxiliary servers, stop each appropriate auxiliary server [Aux2].

Log in to the auxiliary server station as the nsp user.
Open a console window.
Enter the following:

bash$ /opt/nsp/nfmp/auxserver/nms/bin/auxnmsserver.bash auxstop ↵

The auxiliary server stops.

Disable database redundancy

Disable the main database failover and switchover functions.

Log in to the primary main server station [Main1] as the nsp user.
Open a console window.
Enter the following to navigate to the main server configuration directory:

bash$ cd /opt/nsp/nfmp/server/nms/config ↵
Make a backup copy of the nms-server.xml file.
Open the nms-server.xml file with a plain-text editor, for example, vi.
Locate the section that begins with the following tag:

<db
Locate the following line in the section:

host="address"
Ensure that the address value in the line is the IP address of main database [DB1].
Locate the following line in the section:

database="instance_name"
Ensure that the instance_name value is the instance name of main database [DB1].
Edit the following line in the section that reads:

redundancyEnabled="true"

to read:

redundancyEnabled="false"
Save and close the nms-server.xml file.
Enter the following:

bash$ /opt/nsp/nfmp/server/nms/bin/nmsserver.bash read_config ↵

The main server puts the change into effect, and database redundancy is disabled.

Upgrade standby main database [DB2]

Note: After the upgrade, the station is the new primary main database station.

Open a console window.

Stop and disable the Oracle proxy and main database services.

Enter the following to stop the Oracle proxy:

# systemctl stop nfmp-oracle-proxy.service ↵
Enter the following to disable the automatic Oracle proxy startup:

# systemctl disable nfmp-oracle-proxy.service ↵
Enter the following to stop the main database:

# systemctl stop nfmp-main-db.service ↵
Enter the following to disable the automatic database startup:

# systemctl disable nfmp-main-db.service ↵

If analytics aggregations are enabled, perform the following steps to disable all aggregation rules.

Note: Disabling analytics aggregation during a redundant system upgrade prevents the duplication of aggregation data in the NFM-P database, but does not cause the loss of any aggregation data.

Upon startup, if a primary main server detects that the most recent aggregation data is not current, the server performs the interim aggregations. If aggregation is enabled during a redundant upgrade, the original primary main server creates aggregations while the standby main server is upgraded. In such a case, after the standby main server starts as the new primary main server, the server may perform aggregations that are duplicates of the aggregations performed by the original primary main server.

The required aggregation rules are automatically enabled on the new primary main server, so the server performs the interim aggregations upon startup. If aggregation is disabled at the start of a redundant upgrade, no aggregation duplication occurs.

Open an NFM-P GUI client.
Choose Tools→Analytics→Aggregation Manager from the NFM-P main menu. The Aggregation Manager form opens.
Click Search. The aggregation rules are listed.
Click on the Enable Aggregation column to sort the rules so that the rules that have aggregation enabled are at the top of the list.
Select all rules that have a check mark in the Enable Aggregation column.
Click Properties. The Aggregation Rule (multiple instances) [Edit] form opens.
Deselect Enable Aggregation.
Click OK. The Aggregation Rule (multiple instances) [Edit] form closes.
Click OK to save your changes and close the Aggregation Manager form.
Close the NFM-P GUI client.

Perform the following steps.

Perform To apply a RHEL update to an NSP image-based OSon the main database station.
Open the /etc/fstab file using a plain-text editor such as vi.
Locate the tmpfs file system entry.
Remove the noexec option so that the entry reads as follows:

tmpfs /dev/shm tmpfs nodev 0 0
Save and close the /etc/fstab file.
Enter the following to remount the /dev/shm partition:

# mount -o remount /dev/shm ↵

Note: After the upgrade, the station is the new primary main database station.

Perform one of the following.

If the main server and database are collocated on one station, perform the following steps.
1. Transfer the following downloaded installation files to an empty directory on the collocated station:
  - nsp-nfmp-oracle-R.r.p-rel.v.rpm
  - nsp-nfmp-main-db-R.r.p-rel.v.rpm
  - nsp-nfmp-nspos-R.r.p.rpm
  - nsp-nfmp-jre-R.r.p-rel.v.rpm
  - nsp-nfmp-config-R.r.p-rel.v.rpm
  - nsp-nfmp-main-server-R.r.p.rpm
  Note: In subsequent steps, the directory is called the NFM-P software directory.
2. You must remove the semvalidator package if it is installed; otherwise, the upgrade is blocked.
  
  Enter the following:
  
  # rpm -q nsp-nfmp-semvalidator ↵
  
  If the package is installed, the following is displayed:
  
  nsp-nfmp-semvalidator-version
  
  If the package is not installed, the following is displayed:
  
  package nsp-nfmp-semvalidator is not installed
3. If the package is installed, enter the following:
  
  # dnf remove nsp-nfmp-semvalidator ↵
  
  The package is removed.
If the main server and database are on separate stations, transfer the following downloaded installation files to an empty directory on the main database station:
- nsp-nfmp-jre-R.r.p-rel.v.rpm
- nsp-nfmp-config-R.r.p-rel.v.rpm
- nsp-nfmp-oracle-R.r.p-rel.v.rpm
- nsp-nfmp-main-db-R.r.p-rel.v.rpm
- nsp-nfmp-nodeexporter-R.r.p-rel.v.rpm, if downloaded
Note: In subsequent steps, the directory is called the NFM-P software directory.

Transfer the following downloaded file to an empty directory on the main database station:

OracleSw_PreInstall.sh

Navigate to the directory that contains the OracleSw_PreInstall.sh file.

Enter the following:

# chmod +x OracleSw_PreInstall.sh ↵

Enter the following:

# ./OracleSw_PreInstall.sh ↵

Note: A default value is displayed in brackets []. To accept the default, press ↵.

Note: If you specify a value other than the default, you must record the value for use when the OracleSw_PreInstall.sh script is run during a software upgrade, or when the Oracle management user information is required by technical support.

The following prompt is displayed:

This script will prepare the system for an upgrade to NFM-P Version R.r Rn.

Do you want to continue? [Yes/No]:

Enter Yes. The following messages and prompt are displayed:

About to validate that the database can be upgraded to release.

Found the database installation directory /opt/nsp/nfmp/db/install.

Existing database version = version

Enter the password for the "SYS" Oracle user (terminal echo is off):

Enter the SYS user password.

The script begins to validate the database records, and displays the following:

Validating the database for upgrade. Please wait ...

If the validation is successful, the following messages and prompt are displayed:

INFO: Database upgrade validation passed.

Creating group group if it does not exist ...

Checking or Creating the Oracle user home directory /opt/nsp/nfmp/oracle19...

Checking user username... usermod: no changes

Changing ownership of the directory /opt/nsp/nfmp/oracle19 to username:group.

About to unlock the UNIX user [username]

Unlocking password for user username.

passwd: Success

Unlocking the UNIX user [username] completed

Do you want to change the password for the user username? [Yes/No]:

Go to Step 21.

If the database contains an invalid item, for example, an NE at a release that the new NFM-P software does not support, the following is displayed and the script exits:

ERROR: Unsupported records found in database. Please remove the following unsupported items first:

Please remove the following unsupported items first:

item_1

item_2

item_n

ERROR: The database cannot be upgraded. Please fix the above errors and re-run this script.

Perform the following steps.

Use an NFM-P GUI client to remove or update the unsupported items, as required. For example, upgrade an unsupported NE to a release that the new software supports.
Run the script again; go to Step 17.

Perform one of the following.

Enter No to retain the current password.
Specify a new password.
1. Enter Yes. The following prompt is displayed:
  
  New Password:
2. Enter a password. The following prompt is displayed:
  
  Re-enter new Password:
3. Re-enter the password. The following is displayed if the password change is successful:
  
  passwd: password successfully changed for user

The following message and prompt are displayed:

Specify whether an NFM-P server will be installed on this workstation.

The database memory requirements will be adjusted to account for the additional load.

Will the database co-exist with an NFM-P server on this workstation [Yes/No]:

Enter Yes or No, as required.

Messages like the following are displayed as the script execution completes:

INFO: About to remove kernel parameters set by a previous run of this script from /etc/sysctl.conf

INFO: Completed removing kernel parameters set by a previous run of this script from /etc/sysctl.conf

INFO: About to set kernel parameters in /etc/sysctl.conf...

INFO: Completed setting kernel parameters in /etc/sysctl.conf...

INFO: About to change the current values of the kernel parameters

INFO: Completed changing the current values of the kernel parameters

INFO: About to remove ulimit parameters set by a previous run of this script from /etc/security/limits.conf

INFO: Completed removing ulimit parameters set by a previous run of this script from /etc/security/limits.conf

INFO: About to set ulimit parameters in etc/security/limits.conf...

INFO: Completed setting ulimit parameters in /etc/security/limits.conf...

INFO: Completed running Oracle Pre-Install Tasks

When the script execution is complete, enter the following to reboot the main database station:

# systemctl reboot ↵

The station reboots.

When the reboot is complete, log in to the main database [DB2] station as the root user.

Open a console window.

Navigate to the NFM-P software directory.

Note: Ensure that the directory contains only the installation files.

Enter the following:

# chmod +x * ↵

Enter the following:

# dnf install *.rpm ↵

The dnf utility resolves any package dependencies, and displays the following prompt:

Total size: nn G

Installed size: nn G

Is this ok [y/d/N]:

Enter y. The following and the installation status are displayed as each package is installed:

Downloading Packages:

Running transaction check

Transaction check succeeded.

Running transaction test

Transaction test succeeded.

Running transaction

The package installation is complete when the following is displayed:

Complete!

Enter the following to upgrade the database:

Note: A database upgrade takes considerable time that varies, depending on the release from which you are upgrading.

# samupgradeDb ↵

The following prompt is displayed:

Enter the password for the "SAMUSER" database user (terminal echo is off):

Enter the database user password.

The database upgrade begins, and messages like the following are displayed:

Validating...

Validation succeeded.

Upgrade log is /opt/nsp/nfmp/db/install/NFM-P_Main_Database.upgrade.timestamp.stdout.txt

timestamp working..

timestamp Performing Step 1 of n - Initializing ...

timestamp Performing Step n of n - Finalizing ...

timestamp Database upgrade was successful

The database upgrade is complete when the following is displayed:

DONE

Verify the database configuration and create the database.

Note: This main database [DB1] is the new primary main database.

Enter the following:

# samconfig -m db ↵

The following is displayed:

Start processing command line inputs...

<db>
Enter the following:

<db> show-detail ↵

The database configuration is displayed.
Review each parameter to ensure that the value is correct; see NFM-P samconfig utility for information about using samconfig.
Configure one or more parameters, if required, and then enter back ↵.
Enter the following to apply the configuration and create the database:

<db> apply ↵

The configuration is applied, and the database creation begins.
When the database creation is complete, enter the following:

<db> exit ↵

The samconfig utility closes.

Start PKI server

Start the PKI server, regardless of whether you are using the automated or manual TLS configuration method; perform To configure and enable a PKI server.

Note: The PKI server is required for internal system configuration purposes.

Upgrade standby main server [Main2]

If the [Main2] main server and database are on separate stations, and the [Main2] main server is deployed in a VM created using an NSP RHEL OS disk image, perform To apply a RHEL update to an NSP image-based OS on the standby [Main2] main server station.

Note: After the upgrade, the station is the new primary main server station.

Open a console window.

Perform one of the following.

If the main server and database are collocated on one station, go to Step 43.
If the main server and database are on separate stations, transfer the following downloaded installation files to an empty directory on the main server station:
- nsp-nfmp-nspos-R.r.p.rpm
- nsp-nfmp-jre-R.r.p-rel.v.rpm
- nsp-nfmp-config-R.r.p-rel.v.rpm
- nsp-nfmp-main-server-R.r.p.rpm
- nsp-nfmp-nodeexporter-R.r.p-rel.v.rpm, if downloaded
Note: In subsequent steps, the directory is called the NFM-P software directory.

You must remove the semvalidator package if it is installed; otherwise, the upgrade is blocked.

Perform the following steps.

Enter the following:

# rpm -q nsp-nfmp-semvalidator ↵

If the package is installed, the following is displayed:

nsp-nfmp-semvalidator-version

If the package is not installed, the following is displayed:

package nsp-nfmp-semvalidator is not installed
If the package is installed, enter the following:

# dnf remove nsp-nfmp-semvalidator ↵

The package is removed.

Navigate to the NFM-P software directory.

Enter the following:

# chmod +x * ↵

Enter the following:

# dnf install *.rpm ↵

The dnf utility resolves any package dependencies, and displays the following prompt:

Total size: nn G

Installed size: nn G

Is this ok [y/d/N]:

Enter y. The following and the installation status are displayed as each package is installed:

Downloading Packages:

Running transaction check

Transaction check succeeded.

Running transaction test

Transaction test succeeded.

Running transaction

The package installation is complete when the following is displayed:

Complete!

Configure new primary main server [Main2]

Enter the following; see NFM-P samconfig utility for information about using samconfig:

Note: Regardless of whether you intend to modify the main server configuration, you must apply the main server configuration, as described in the following steps.

# samconfig -m main ↵

The following is displayed:

Start processing command line inputs...

<main>

Enter the following:

<main> configure ↵

The prompt changes to <main configure>.

To apply a new or updated NFM-P license, enter the following:

Note: You cannot start a main server unless the main server configuration includes a current and valid license. You can use samconfig to specify the license file in this step, or later import the license, as described in the NSP System Administrator Guide.

<main configure> license license_file back ↵

where license_file is the path and file name of the NSP license bundle

Verify the main server configuration.

Enter the following:

<main configure> show ↵

The main server configuration is displayed.
Review each parameter to ensure that the value is correct; see NFM-P samconfig utility for information about using the samconfig utility.
Configure one or more parameters, if required.

Note: The NFM-P uses the database backup settings to initialize the database during installation only. To change the backup settings after installation, you must use the Database Manager form in the NFM-P client GUI, as described in the NSP System Administrator Guide.
When you are certain that the configuration is correct, enter the following:

<main configure> back ↵

The prompt changes to <main>.

Enter the following:

<main> apply ↵

The configuration is applied.

Enter the following:

<main> exit ↵

The samconfig utility closes.

To enable mTLS for internal Kafka authentication using two-way TLS, perform the following steps.

Note: Enabling mTLS for internal Kafka authentication is supported only in an NSP deployment that uses separate interfaces for internal and client communication.

Enter the following:

# samconfig -m main ↵

The following is displayed:

Start processing command line inputs...

<main>
Enter the following:

# configure nspos mtls-kafka-enabled back ↵
Enter the following:

<main> apply ↵

The configuration is applied.
Enter the following:

<main> exit ↵

The samconfig utility closes.

Enable Windows Active Directory access

If you intend to use Windows Active Directory, or AD, for single sign-on client access, you must configure LDAP remote authentication for AD; otherwise,go to Step 59.

Open the following file as a reference for use in subsequent steps:

/opt/nsp/os/install/examples/config.yml

Note: Consider the following.

The NFM-P does not assign a default user group to users of a remote authentication source that you define for Windows AD; the authentication source must provide the user group attributes.
Windows AD supports the following LDAP server types for remote authentication:

AD—The user group of an AD user is derived from the group_base_dn attribute in the server configuration; group search filters are not supported.

AUTHENTICATED—The server configuration must include bind credentials; group search filters are supported. After NFM-P initialization, you add the AD server bind credentials to the NSP password vault using the NSP Session Manager RESTCONF API.

Locate the section that begins with the following lines:

# ldap:

# enabled: true

# servers:

# - type: AUTHENTICATED/AD/ANONYMOUS

# url: ldaps://ldap.example.com:636

# security: SSL/STARTTLS/NONE

Open the following file using a plain-text editor such as vi:

/opt/nsp/os/install/config.json

Locate the section that begins with the following line:

"sso": {

The section has one subsection for each type of SSO access.

Note: You can enable multiple remote authentication methods such as LDAP and RADIUS in the config.json file, or by using the NFM-P GUI. Using the GUI also allows you to specify the order in which the methods are tried during login attempts; however, no ordering is applied to multiple methods enabled in the config.json file.

In the sso section, create an ldap subsection as shown below using the parameter names from the ldap section of config.yml and the required values for your configuration.

The following example shows the LDAP configuration for two AD servers:

    "ldap": {

      "enabled": true,

      "servers": [

          "type": "auth_type",

          "url": "ldaps://server1:port",

          "server1_parameter_1": "value",

          "server1_parameter_2": "value",

          "server1_parameter_n": "value",

},

          "type": "auth_type",

          "url": "ldaps://server2:port",

          "server2_parameter_1": "value",

          "server2_parameter_2": "value",

          "server2_parameter_n": "value",

},

}]

where auth_type is AD or AUTHENTICATED

Save and close the files.

Enter the following:

# samconfig -m main ↵

The following is displayed:

Start processing command line inputs...

<main>

Enter the following:

<main> apply ↵

The AD LDAP configuration is applied.

Enter the following:

<main> exit ↵

The samconfig utility closes.

Configure WS-NOC integration

If the NFM-P is integrated with a WS-NOC system, open the following file with a plain-text editor such as vi:

/opt/nsp/os/install/examples/config.json

Otherwise, go to Step 69.

Copy the following section:

"nfmt": {

"primary_ip": "",

"standby_ip": "",

"username": "",

"password": "",

"cert_provided": false

},

Close the file.

Open the following file with a plain-text editor such as vi:

/opt/nsp/os/install/config.json

Paste in the copied section.

Configure the required parameters to enable the WS-NOC integration:

primary_ip—the primary WS-NOC server IP address
standby_ip—the standby WS-NOC server IP address
username—the username required for WS-NOC access
password—the password required for WS-NOC access
cert_provided—whether a TLS certificate is used

Save and close the file.

Enter the following:

# samconfig -m main ↵

The following is displayed:

Start processing command line inputs...

<main>

Enter the following:

<main> apply ↵

The configuration is applied.

Enter the following:

<main> exit ↵

The samconfig utility closes.

Stop NSP analytics servers, NSP Flow Collectors

If the system includes one or more NSP analytics servers, stop each analytics server.

Log in to the analytics server station as the nsp user.
Open a console window.
Enter the following:

bash$ /opt/nsp/analytics/bin/AnalyticsAdmin.sh stop ↵

The following is displayed:

Stopping Analytics Application

When the analytics server is completely stopped, the following message is displayed:

Analytics Application is not running

If the system includes one or more NSP Flow Collector Controllers and Flow Collectors, stop each NSP Flow Collector Controller.

Note: If the NSP Flow Collector Controller is collocated on a station with an NSP Flow Collector, stopping the NSP Flow Collector Controller also stops the Flow Collector.

Log in to the NSP Flow Collector Controller station as the nsp user.
Open a console window.
Enter the following:

bash$ /opt/nsp/flow/fcc/bin/flowCollectorController.bash stop ↵

The NSP Flow Collector Controller stops.

If the system includes one or more NSP Flow Collectors that are not collocated on a station with a Flow Collector Controller, stop each such NSP Flow Collector.

Log in to the NSP Flow Collector station as the nsp user.
Open a console window.
Enter the following:

bash$ /opt/nsp/flow/fc/bin/flowCollector.bash stop ↵

The NSP Flow Collector stops.

Upgrade auxiliary servers [Aux2]

If the system includes auxiliary servers, perform To upgrade a Release 22.9 or later NFM-P auxiliary server on each appropriate auxiliary server station [Aux2].

Upgrade standby auxiliary database cluster

Perform To upgrade a geo-redundant Release 22.9 or later auxiliary database on the standby auxiliary database cluster.

Stop and disable original primary main server [Main1]

Stop the original primary main server.

Note: This step marks the beginning of the network management outage.

Log in to the original primary main server station [Main1] as the nsp user.
Open a console window.
Enter the following:

bash$ cd /opt/nsp/nfmp/server/nms/bin ↵
Enter the following:

bash$ ./nmsserver.bash stop ↵
Enter the following:

bash$ ./nmsserver.bash appserver_status ↵

The server status is displayed; the server is fully stopped if the status is the following:

Application Server is stopped

If the server is not fully stopped, wait five minutes and then repeat this step. Do not perform the next step until the server is fully stopped.
Enter the following to switch to the root user:

bash$ su ↵

Disable the automatic main server startup so that the main server does not start in the event of a power disruption during the upgrade.

Enter the following:

# systemctl disable nspos-nspd.service ↵
Enter the following:

# systemctl disable nfmp-main-config.service ↵
Enter the following:

# systemctl disable nfmp-main.service ↵

Upgrade NSP Flow Collector Controllers, Flow Collectors

If the system includes one or more NSP Flow Collectors, upgrade each NSP Flow Collector Controller and Flow Collector as described in NSP Flow Collector and Flow Collector Controller upgrade from Release 22.9 or later.

Stop auxiliary servers [Aux1]

If the system includes auxiliary servers, perform the following steps on each [Aux1] auxiliary server station.

Log in to the auxiliary server station as the nsp user.
Open a console window.
Enter the following:

bash$ /opt/nsp/nfmp/auxserver/nms/bin/auxnmsserver.bash auxstop ↵

The auxiliary server stops.

Stop original primary main database [DB1]

Open a console window.

Stop and disable the Oracle proxy and main database services.

Enter the following to stop the Oracle proxy:

# systemctl stop nfmp-oracle-proxy.service ↵
Enter the following to disable the automatic Oracle proxy startup:

# systemctl disable nfmp-oracle-proxy.service ↵
Enter the following to stop the main database:

# systemctl stop nfmp-main-db.service ↵
Enter the following to disable the automatic database startup:

# systemctl disable nfmp-main-db.service ↵

Perform the following steps.

Perform To apply a RHEL update to an NSP image-based OSon the main database station.
Open the /etc/fstab file using a plain-text editor such as vi.
Locate the tmpfs file system entry.
Remove the noexec option so that the entry reads as follows:

tmpfs /dev/shm tmpfs nodev 0 0
Save and close the /etc/fstab file.
Enter the following to remount the /dev/shm partition:

# mount -o remount /dev/shm ↵

Stop former primary auxiliary database cluster

If the system includes an auxiliary database, perform the following steps on one station in the former primary auxiliary database cluster.

Log in as the root user.
Open a console window.
Enter the following:

# cd /opt/nsp/nfmp/auxdb/install/bin ↵
Enter the following to stop the auxiliary database:

# ./auxdbAdmin.sh stop ↵
Enter the following to display the auxiliary database status:

# ./auxdbAdmin.sh status ↵

Information like the following is displayed:

Database status

Node | Host | State | Version | DB

------------+---------------+-------+---------+-------

node_1 | internal_IP_1 | STATE | version | db_name

node_2 | internal_IP_2 | STATE | version | db_name

.

.

.

node_n | internal_IP_n | STATE | version | db_name

Output captured in log_file

The cluster is stopped when each STATE entry reads DOWN.
Repeat substep 5 periodically until the cluster is stopped.

Note: You must not proceed to the next step until the cluster is stopped.

Integrate NFM-P and NSP cluster

If you are upgrading an NFM-P system that is not currently integrated with an NSP cluster, perform procedure To add an independent NFM-P to an existing NSP deployment.

Note: Performing the procedure also starts the main server.

Start new primary main server [Main2]

CAUTION

Service Disruption

The new primary database [DB2] must be upgraded and running before you start the new primary main server [Main2], or the main server initialization may fail.

If you perform the new primary main server and database upgrades concurrently, do not perform this step until the database upgrade is complete.

CAUTION

Service Disruption

An NFM-P system upgrade is not complete until each main server performs crucial post-upgrade tasks during initialization.

Before you attempt an operation that requires a server shutdown, you must ensure that each main server is completely initialized, or the operation fails.

Start the new primary main server [Main2].

Note: You cannot start a main server unless the main server configuration includes a current and valid license. You can use samconfig to specify the license file, or import a license, as described in the NSP System Administrator Guide.

Log in as the nsp user on the new primary main server station [Main2].
Enter the following:

bash$ cd /opt/nsp/nfmp/server/nms/bin ↵
Enter the following:

bash$ ./nmsserver.bash start ↵
Enter the following:

bash$ ./nmsserver.bash appserver_status ↵

The server status is displayed; the server is fully initialized if the status is the following:

Application Server process is running. See nms_status for more detail.

If the server is not fully initialized, wait five minutes and then repeat this step. Do not perform the next step until the server is fully initialized.

Note: This marks the end of the network management outage.

If you have enabled Windows Active Directory access using the AUTHENTICATED type of LDAP server, use the NSP Session Manager RESTCONF API to add the LDAP server bind credentials; see the Network Developer Portal for information.

Specify the memory requirement for GUI clients based on the type of network that the NFM-P is to manage.

Enter the following:

bash$ ./nmsdeploytool.bash clientmem -option ↵

where option is one of the following:
- m—medium, for management of limited-scale network
- l—large, for a network of 15 000 or more NEs
Record the setting, which is not preserved through an upgrade, for future use.
Enter the following to commit the configuration change:

bash$ ./nmsdeploytool.bash deploy ↵

Start auxiliary servers [Aux2]

If the NFM-P system includes auxiliary servers, start each appropriate auxiliary server [Aux2].

Log in to the auxiliary server station as the nsp user.
Open a console window.
Enter the following:

bash$ cd /opt/nsp/nfmp/auxserver/nms/bin ↵
Enter the following:

bash$ ./auxnmsserver.bash auxstart ↵

The auxiliary server starts.

Activate upgraded former standby auxiliary database cluster

If the system does not include an auxiliary database, go to Step 91.

Perform the following steps on each station in the upgraded former standby auxiliary database cluster.

Log in as the root user on the station.
Open a console window.
Enter the following sequence of commands to enable the database services:

systemctl enable nspos-auxdb.service

systemctl enable nspos-auxdbproxy.service

systemctl enable vertica_agent.service

systemctl enable verticad.service

The services are enabled.
Enter the following to start the database proxy:

# systemctl start nspos-auxdbproxy.service ↵

The proxy starts.

Issue the following RESTCONF API call to activate the former standby auxiliary database cluster, after which the cluster assumes the primary role:

Note: In order to issue a RESTCONF API call, you require a token; see this tutorial on the Network Developer Portal for information.

POST https://{{address}}/RESTCONF/data/auxdb:/clusters/cluster=cluster_N/activate

where

address is the advertised address of the primary NSP cluster

N is the auxiliary database cluster number

The following is the request body:

{

"auxdb:input" : {

"force": true

}

Perform NSP analytics server migration to NSP cluster

If the system includes one or more NSP analytics servers, move the analytics-server functions to the NSP cluster, as described in NSP analytics server migration from Release 22.9 or later.

Enable GUI client

You require an NFM-P GUI client to complete the procedure; see the following for information:

Note: A client delegate server installation typically takes more time than the other options. A single-user client or client delegate server upgrade is recommended if your maintenance period is limited.

Test upgraded system using GUI client

When the new primary main server [Main2] is started, use a newly installed or upgraded GUI client to perform sanity testing of the new primary main server and database.

Note: To back out of the upgrade and return the original primary main server [Main1] and database [DB1] to service, you can do so by stopping the new primary main server [Main2] and database [DB2] and restarting the original primary main server [Main1] and database [DB1].

Uninstall original primary database [DB1]

Enter the following to uninstall the original primary main database:

# dnf remove nsp-nfmp-main-db nsp-nfmp-oracle ↵

The dnf utility resolves any dependencies and displays the following prompt:

Installed size: nn G

Is this ok [y/N]:

Enter y. The following is displayed as the packages are removed:

Downloading Packages:

Running transaction check

Transaction check succeeded.

Running transaction test

Transaction test succeeded.

Running transaction

Uninstalling the NFM-P package...

As each package removal completes, the following is displayed:

Complete!

Install new standby main database [DB1]

Note: After the upgrade, the station is the new standby main database station.

Perform one of the following.

If the main server and database are collocated on one station, perform the following steps.
1. Transfer the following downloaded installation files to an empty directory on the collocated station:
  - nsp-nfmp-oracle-R.r.p-rel.v.rpm
  - nsp-nfmp-main-db-R.r.p-rel.v.rpm
  - nsp-nfmp-nspos-R.r.p.rpm
  - nsp-nfmp-jre-R.r.p-rel.v.rpm
  - nsp-nfmp-config-R.r.p-rel.v.rpm
  - nsp-nfmp-main-server-R.r.p.rpm
  Note: In subsequent steps, the directory is called the NFM-P software directory.
2. You must remove the semvalidator package if it is installed; otherwise, the upgrade is blocked.
  
  Enter the following:
  
  # rpm -q nsp-nfmp-semvalidator ↵
  
  If the package is installed, the following is displayed:
  
  nsp-nfmp-semvalidator-version
  
  If the package is not installed, the following is displayed:
  
  package nsp-nfmp-semvalidator is not installed
3. If the package is installed, enter the following:
  
  # dnf remove nsp-nfmp-semvalidator ↵
  
  The package is removed.
If the main server and database are on separate stations, transfer the following downloaded installation files to an empty directory on the main database station:
- nsp-nfmp-jre-R.r.p-rel.v.rpm
- nsp-nfmp-config-R.r.p-rel.v.rpm
- nsp-nfmp-oracle-R.r.p-rel.v.rpm
- nsp-nfmp-main-db-R.r.p-rel.v.rpm
- nsp-nfmp-nodeexporter-R.r.p-rel.v.rpm, if downloaded
Note: In subsequent steps, the directory is called the NFM-P software directory.

Transfer the following downloaded file to an empty directory on the main database station:

OracleSw_PreInstall.sh

Open a console window.

100

Navigate to the directory that contains the OracleSw_PreInstall.sh file.

101

Enter the following:

# chmod +x OracleSw_PreInstall.sh ↵

102

Enter the following:

# ./OracleSw_PreInstall.sh ↵

Note: A default value is displayed in brackets []. To accept the default, press ↵.

The following prompt is displayed:

This script will prepare the system for a new install/restore of an NFM-P Version R.r Rn database.

Do you want to continue? [Yes/No]:

103

Enter Yes. The following prompt is displayed:

Enter the Oracle dba group name [group]:

104

Press ↵ to accept the default.

The following messages and prompt are displayed:

Creating group group if it does not exist...

WARNING: Group group already exists locally.

Do you want to use the existing group? [Yes/No]:

105

Enter Yes.

The following message and prompt are displayed:

The user [username] for the group [group] already exists locally.

Do you want to use the existing user? [Yes/No]:

106

Enter Yes.

The following messages and prompt are displayed:

Checking or Creating the Oracle user home directory /opt/nsp/nfmp/oracle19...

Checking user username...

WARNING: Oracle user with the specified name already exists locally.

Redefining the primary group and home directory of user username ... usermod: no changes

Changing ownership of the directory /opt/nsp/nfmp/oracle19 to username:group.

About to unlock the UNIX user [username]

Unlocking password for user username.

passwd: Success

Unlocking the UNIX user [username] completed

Do you want to change the password for the user username? [Yes/No]:

107

Perform one of the following.

If you did not change the password during the upgrade of the original standby database, enter No.
If you changed the password during the upgrade of the original standby database, perform the following steps.
1. Enter Yes. The following prompt is displayed:
  
  New Password:
2. Enter a password. The following prompt is displayed:
  
  Re-enter new Password:
3. Re-enter the password. The following is displayed if the password change is successful:
  
  passwd: password successfully changed for user

The following message and prompt are displayed:

Specify whether an NFM-P nserver will be installed on this workstation.

The database memory requirements will be adjusted to account for the additional load.

Will the database co-exist with an NFM-P server on this workstation [Yes/No]:

108