To upgrade a redundant Release 22.9 or later NFM-P system

Description

The following steps describe how to upgrade a collocated or distributed Release 22.9 or later main database and main server in a redundant deployment. The steps include links to procedures for installing and upgrading optional NFM-P components.

Ensure that you record the information that you specify, for example, directory names, passwords, and IP addresses.

Note: You require the following user privileges:

on each server station in the system — root, nsp
on each main database station — root

Note: The following RHEL CLI prompts in command lines denote the active user, and are not to be included in typed commands:

# —root user
bash$ —nsp user

Steps

Check pre-upgrade disk space

As part of the trial upgrade on a lab system in advance of a live upgrade, you must ensure that the available disk capacity on each NFM-P component remains within tolerance.

Note: If the disk usage on an NFM-P partition approaches or exceeds 80% after the trial upgrade, you may need to add disk capacity before you attempt the upgrade on a live system.

Perform the following steps on each of the following stations:

main server
auxiliary server
main database
auxiliary database

Log in to the station as the root user.
Open a console window.
Enter the following:

# df -kh ↵

The usage information for each partition is displayed.
Record the information for each NFM-P partition; see the tables in Chapter 2, NSP disk setup and partitioning for the partition names and required capacities.

Stop and disable standby main server [Main2]

Open a GUI client to monitor the network during the upgrade.

Stop the standby main server.

Log in to the standby main server station as the nsp user.
Open a console window.
Enter the following:

bash$ cd /opt/nsp/nfmp/server/nms/bin ↵
Enter the following:

bash$ ./nmsserver.bash stop ↵
Enter the following:

bash$ ./nmsserver.bash appserver_status ↵

The server status is displayed; the server is fully stopped if the status is the following:

Application Server is stopped

If the server is not fully stopped, wait five minutes and then repeat this step. Do not perform the next step until the server is fully stopped.
Enter the following to switch to the root user:

bash$ su ↵
If the NFM-P is not part of a shared-mode NSP deployment, enter the following to display the nspOS service status:

# nspdctl status ↵

Information like the following is displayed.

Mode:     DR

Role:     redundancy_role

DC-Role:  dc_role

DC-Name:  dc_name

Registry: IP_address:port

State:    stopped

Uptime:   0s

SERVICE           STATUS

service_a         inactive

service_b         inactive

service_c         inactive

You must not proceed to the next step until all NSP services are stopped; if the State is not ‘stopped’, or the STATUS indicator of each listed service is not ‘inactive’, repeat this substep.

Disable the automatic main server startup so that the main server does not start in the event of a power disruption during the upgrade.

Enter the following:

# systemctl disable nspos-nspd.service ↵
Enter the following:

# systemctl disable nfmp-main-config.service ↵
Enter the following:

# systemctl disable nfmp-main.service ↵

Stop auxiliary servers [Aux2]

If the NFM-P system includes auxiliary servers, stop each appropriate auxiliary server [Aux2].

Log in to the auxiliary server station as the nsp user.
Open a console window.
Enter the following:

bash$ /opt/nsp/nfmp/auxserver/nms/bin/auxnmsserver.bash auxstop ↵

The auxiliary server stops.

Disable database redundancy

Disable the main database failover and switchover functions.

Log in to the primary main server station [Main1] as the nsp user.
Open a console window.
Enter the following to navigate to the main server configuration directory:

bash$ cd /opt/nsp/nfmp/server/nms/config ↵
Make a backup copy of the nms-server.xml file.
Open the nms-server.xml file with a plain-text editor, for example, vi.
Locate the section that begins with the following tag:

<db
Locate the following line in the section:

host="address"
Ensure that the address value in the line is the IP address of main database [DB1].
Locate the following line in the section:

database="instance_name"
Ensure that the instance_name value is the instance name of main database [DB1].
Edit the following line in the section that reads:

redundancyEnabled="true"

to read:

redundancyEnabled="false"
Save and close the nms-server.xml file.
Enter the following:

bash$ /opt/nsp/nfmp/server/nms/bin/nmsserver.bash read_config ↵

The main server puts the change into effect, and database redundancy is disabled.

Upgrade standby main database [DB2]

Note: After the upgrade, the station is the new primary main database station.

Open a console window.

Stop and disable the Oracle proxy and main database services.

Enter the following to stop the Oracle proxy:

# systemctl stop nfmp-oracle-proxy.service ↵
Enter the following to disable the automatic Oracle proxy startup:

# systemctl disable nfmp-oracle-proxy.service ↵
Enter the following to stop the main database:

# systemctl stop nfmp-main-db.service ↵
Enter the following to disable the automatic database startup:

# systemctl disable nfmp-main-db.service ↵

If analytics aggregations are enabled, perform the following steps to disable all aggregation rules.

Note: Disabling analytics aggregation during a redundant system upgrade prevents the duplication of aggregation data in the NFM-P database, but does not cause the loss of any aggregation data.

Upon startup, if a primary main server detects that the most recent aggregation data is not current, the server performs the interim aggregations. If aggregation is enabled during a redundant upgrade, the original primary main server creates aggregations while the standby main server is upgraded. In such a case, after the standby main server starts as the new primary main server, the server may perform aggregations that are duplicates of the aggregations performed by the original primary main server.

The required aggregation rules are automatically enabled on the new primary main server, so the server performs the interim aggregations upon startup. If aggregation is disabled at the start of a redundant upgrade, no aggregation duplication occurs.

Open an NFM-P GUI client.
Choose Tools→Analytics→Aggregation Manager from the NFM-P main menu. The Aggregation Manager form opens.
Click Search. The aggregation rules are listed.
Click on the Enable Aggregation column to sort the rules so that the rules that have aggregation enabled are at the top of the list.
Select all rules that have a check mark in the Enable Aggregation column.
Click Properties. The Aggregation Rule (multiple instances) [Edit] form opens.
Deselect Enable Aggregation.
Click OK. The Aggregation Rule (multiple instances) [Edit] form closes.
Click OK to save your changes and close the Aggregation Manager form.
Close the NFM-P GUI client.

Perform the following steps.

Perform To apply a RHEL update to an NSP image-based OSon the main database station.
Open the /etc/fstab file using a plain-text editor such as vi.
Locate the tmpfs file system entry.
Remove the noexec option so that the entry reads as follows:

tmpfs /dev/shm tmpfs nodev 0 0
Save and close the /etc/fstab file.
Enter the following to remount the /dev/shm partition:

# mount -o remount /dev/shm ↵

Note: After the upgrade, the station is the new primary main database station.

Perform one of the following.

If the main server and database are collocated on one station, perform the following steps.
1. Transfer the following downloaded installation files to an empty directory on the collocated station:
  - nsp-nfmp-oracle-R.r.p-rel.v.rpm
  - nsp-nfmp-main-db-R.r.p-rel.v.rpm
  - nsp-nfmp-nspos-R.r.p.rpm
  - nsp-nfmp-jre-R.r.p-rel.v.rpm
  - nsp-nfmp-config-R.r.p-rel.v.rpm
  - nsp-nfmp-main-server-R.r.p.rpm
  Note: In subsequent steps, the directory is called the NFM-P software directory.
2. You must remove the semvalidator package if it is installed; otherwise, the upgrade is blocked.
  
  Enter the following:
  
  # rpm -q nsp-nfmp-semvalidator ↵
  
  If the package is installed, the following is displayed:
  
  nsp-nfmp-semvalidator-version
  
  If the package is not installed, the following is displayed:
  
  package nsp-nfmp-semvalidator is not installed
3. If the package is installed, enter the following:
  
  # dnf remove nsp-nfmp-semvalidator ↵
  
  The package is removed.
If the main server and database are on separate stations, transfer the following downloaded installation files to an empty directory on the main database station:
- nsp-nfmp-jre-R.r.p-rel.v.rpm
- nsp-nfmp-config-R.r.p-rel.v.rpm
- nsp-nfmp-oracle-R.r.p-rel.v.rpm
- nsp-nfmp-main-db-R.r.p-rel.v.rpm
- nsp-nfmp-nodeexporter-R.r.p-rel.v.rpm, if downloaded
Note: In subsequent steps, the directory is called the NFM-P software directory.

Transfer the following downloaded file to an empty directory on the main database station:

OracleSw_PreInstall.sh

Navigate to the directory that contains the OracleSw_PreInstall.sh file.

Enter the following:

# chmod +x OracleSw_PreInstall.sh ↵

Enter the following:

# ./OracleSw_PreInstall.sh ↵

Note: A default value is displayed in brackets []. To accept the default, press ↵.

Note: If you specify a value other than the default, you must record the value for use when the OracleSw_PreInstall.sh script is run during a software upgrade, or when the Oracle management user information is required by technical support.

The following prompt is displayed:

This script will prepare the system for an upgrade to NFM-P Version R.r Rn.

Do you want to continue? [Yes/No]:

Enter Yes. The following messages and prompt are displayed:

About to validate that the database can be upgraded to release.

Found the database installation directory /opt/nsp/nfmp/db/install.

Existing database version = version

Enter the password for the "SYS" Oracle user (terminal echo is off):

Enter the SYS user password.

The script begins to validate the database records, and displays the following:

Validating the database for upgrade. Please wait ...

If the validation is successful, the following messages and prompt are displayed:

INFO: Database upgrade validation passed.

Creating group group if it does not exist ...

Checking or Creating the Oracle user home directory /opt/nsp/nfmp/oracle19...

Checking user username... usermod: no changes

Changing ownership of the directory /opt/nsp/nfmp/oracle19 to username:group.

About to unlock the UNIX user [username]

Unlocking password for user username.

passwd: Success

Unlocking the UNIX user [username] completed

Do you want to change the password for the user username? [Yes/No]:

Go to Step 21.

If the database contains an invalid item, for example, an NE at a release that the new NFM-P software does not support, the following is displayed and the script exits:

ERROR: Unsupported records found in database. Please remove the following unsupported items first:

Please remove the following unsupported items first:

item_1

item_2

item_n

ERROR: The database cannot be upgraded. Please fix the above errors and re-run this script.

Perform the following steps.

Use an NFM-P GUI client to remove or update the unsupported items, as required. For example, upgrade an unsupported NE to a release that the new software supports.
Run the script again; go to Step 17.

Perform one of the following.

Enter No to retain the current password.
Specify a new password.
1. Enter Yes. The following prompt is displayed:
  
  New Password:
2. Enter a password. The following prompt is displayed:
  
  Re-enter new Password:
3. Re-enter the password. The following is displayed if the password change is successful:
  
  passwd: password successfully changed for user

The following message and prompt are displayed:

Specify whether an NFM-P server will be installed on this workstation.

The database memory requirements will be adjusted to account for the additional load.

Will the database co-exist with an NFM-P server on this workstation [Yes/No]:

Enter Yes or No, as required.

Messages like the following are displayed as the script execution completes:

INFO: About to remove kernel parameters set by a previous run of this script from /etc/sysctl.conf

INFO: Completed removing kernel parameters set by a previous run of this script from /etc/sysctl.conf

INFO: About to set kernel parameters in /etc/sysctl.conf...

INFO: Completed setting kernel parameters in /etc/sysctl.conf...

INFO: About to change the current values of the kernel parameters

INFO: Completed changing the current values of the kernel parameters

INFO: About to remove ulimit parameters set by a previous run of this script from /etc/security/limits.conf

INFO: Completed removing ulimit parameters set by a previous run of this script from /etc/security/limits.conf

INFO: About to set ulimit parameters in etc/security/limits.conf...

INFO: Completed setting ulimit parameters in /etc/security/limits.conf...

INFO: Completed running Oracle Pre-Install Tasks

When the script execution is complete, enter the following to reboot the main database station:

# systemctl reboot ↵

The station reboots.

When the reboot is complete, log in to the main database [DB2] station as the root user.

Open a console window.

Navigate to the NFM-P software directory.

Note: Ensure that the directory contains only the installation files.

Enter the following:

# chmod +x * ↵

Enter the following:

# dnf install *.rpm ↵

The dnf utility resolves any package dependencies, and displays the following prompt:

Total size: nn G

Installed size: nn G

Is this ok [y/d/N]:

Enter y. The following and the installation status are displayed as each package is installed:

Downloading Packages:

Running transaction check

Transaction check succeeded.

Running transaction test

Transaction test succeeded.

Running transaction check

The package installation is complete when the following is displayed:

Complete!

Enter the following to upgrade the database:

Note: A database upgrade takes considerable time that varies, depending on the release from which you are upgrading.

# samupgradeDb ↵

The following prompt is displayed:

Enter the password for the "SAMUSER" database user (terminal echo is off):

Enter the database user password.

The database upgrade begins, and messages like the following are displayed:

Validation succeeded.

Upgrade log is /opt/nsp/nfmp/db/install/NFM-P_Main_Database.upgrade.timestamp.stdout.txt

Performing Step 1 of n - Initializing ...........

Performing NFM-P database upgrade will take time...

Executing PreOraUpgrade.sql .............

Executing ShutdownDBUpgrade.sql ....

Executing StartupDB.sql .....

Executing DbJavaReload.sql ................

The database upgrade is complete when the following is displayed:

DONE

Verify the database configuration and create the database.

Note: This main database [DB1] is the new primary main database.

Enter the following:

# samconfig -m db ↵

The following is displayed:

Start processing command line inputs...

<db>
Enter the following:

<db> show-detail ↵

The database configuration is displayed.
Review each parameter to ensure that the value is correct; see NFM-P samconfig utility for information about using samconfig.
Configure one or more parameters, if required, and then enter back ↵.
Enter the following to apply the configuration and create the database:

<db> apply ↵

The configuration is applied, and the database creation begins.
When the database creation is complete, enter the following:

<db> exit ↵

The samconfig utility closes.

Upgrade standby main server [Main2]

If the [Main2] main server and database are on separate stations, and the [Main2] main server is deployed in a VM created using an NSP RHEL OS disk image, perform To apply a RHEL update to an NSP image-based OS on the standby [Main2] main server station.

Note: After the upgrade, the station is the new primary main server station.

Open a console window.

Perform one of the following.

If the main server and database are collocated on one station, go to Step 43.
If the main server and database are on separate stations, transfer the following downloaded installation files to an empty directory on the main server station:
- nsp-nfmp-nspos-R.r.p.rpm
- nsp-nfmp-jre-R.r.p-rel.v.rpm
- nsp-nfmp-config-R.r.p-rel.v.rpm
- nsp-nfmp-main-server-R.r.p.rpm
- nsp-nfmp-nodeexporter-R.r.p-rel.v.rpm, if downloaded
Note: In subsequent steps, the directory is called the NFM-P software directory.

You must remove the semvalidator package if it is installed; otherwise, the upgrade is blocked.

Perform the following steps.

Enter the following:

# rpm -q nsp-nfmp-semvalidator ↵

If the package is installed, the following is displayed:

nsp-nfmp-semvalidator-version

If the package is not installed, the following is displayed:

package nsp-nfmp-semvalidator is not installed
If the package is installed, enter the following:

# dnf remove nsp-nfmp-semvalidator ↵

The package is removed.

Navigate to the NFM-P software directory.

Enter the following:

# chmod +x * ↵

Enter the following:

# dnf install *.rpm ↵

The dnf utility resolves any package dependencies, and displays the following prompt:

Total size: nn G

Installed size: nn G

Is this ok [y/d/N]:

Enter y. The following and the installation status are displayed as each package is installed:

Downloading Packages:

Running transaction check

Transaction check succeeded.

Running transaction test

Transaction test succeeded.

Running transaction check

The package installation is complete when the following is displayed:

Complete!

Start PKI server

Start the PKI server, regardless of whether you are using the automated or manual TLS configuration method; perform To configure and enable a PKI server.

Note: The PKI server is required for internal system configuration purposes.

Configure new primary main server [Main2]

Enter the following; see NFM-P samconfig utility for information about using samconfig:

Note: Regardless of whether you intend to modify the main server configuration, you must apply the main server configuration, as described in the following steps.

# samconfig -m main ↵

The following is displayed:

Start processing command line inputs...

<main>

Enter the following:

<main> configure ↵

The prompt changes to <main configure>.

To apply a new or updated NFM-P license, enter the following:

Note: You cannot start a main server unless the main server configuration includes a current and valid license. You can use samconfig to specify the license file in this step, or later import the license, as described in the NSP System Administrator Guide.

<main configure> license license_file back ↵

where license_file is the path and file name of the NSP license bundle

Verify the main server configuration.

Enter the following:

<main configure> show ↵

The main server configuration is displayed.
Review each parameter to ensure that the value is correct; see NFM-P samconfig utility for information about using the samconfig utility.
Configure one or more parameters, if required.

Note: The NFM-P uses the database backup settings to initialize the database during installation only. To change the backup settings after installation, you must use the Database Manager form in the NFM-P client GUI, as described in the NSP System Administrator Guide.
When you are certain that the configuration is correct, enter the following:

<main configure> back ↵

The prompt changes to <main>.

Enter the following:

<main> apply ↵

The configuration is applied.

Enter the following:

<main> exit ↵

The samconfig utility closes.

If the NFM-P is part of a shared-mode NSP system and you want to enable mTLS for internal Kafka authentication using two-way TLS, perform the following steps.

Note: Enabling mTLS for internal Kafka authentication is supported only in an NSP deployment that uses separate interfaces for internal and client communication.

Note: The parameter you must configure is displayed only if the ip-list parameter is set to a remote address.

Note: The parameter is configurable only if the secure and internal-certs parameters in the nspos section are set to true.

Enter the following:

# samconfig -m main ↵

The following is displayed:

Start processing command line inputs...

<main>
Enter the following:

# configure nspos mtls-kafka-enabled back ↵
Enter the following:

<main> apply ↵

The configuration is applied.
Enter the following:

<main> exit ↵

The samconfig utility closes.

Restore embedded nspOS, independent deployment

In an independent NFM-P deployment, you must restore the embedded Neo4j and PostgreSQL databases. Otherwise, if the NFM-P is integrated with an NSP cluster, go to Step 56.

Enter the following:

# mkdir /opt/nsp/os/backup ↵

Enter the following:

# chown nsp:nsp /opt/nsp/os/backup ↵

Copy the Neo4j and PostgreSQL backup files saved in Step 24 of To prepare for an NFM-P system upgrade from Release 22.9 or later to the /opt/nsp/os/backup directory.

Restore the Neo4j database.

Enter the following:

# cd /opt/nsp/os/install/tools/database ↵
Enter the following:

# ./db-restore.sh --target IP_address ↵

where IP_address is the main server IP address

The following message and prompt are displayed:

Verifying prerequisites...

Starting database restore ...

Backupset file to restore (.tar.gz format):
Enter the following and press ↵:

path/nspos-neo4j_backup_timestamp.tar.gz

where

path is the absolute path of the Neo4j backup file

timestamp is the backup creation time

Note: Neo4j backup files are stored in the following locations on a main server, depending on the backup type:
- scheduled backup—/opt/nsp/os/backup/backupset_n
- manual backup—/opt/nsp/os/backup/manual_timestamp
The following messages and prompt are displayed:

PLAY [all] **************************************************

TASK [dbrestore : Create temporary directory] ***************

changed: [server_IP]

[dbrestore : pause]

Do you want to restore the nspOS Neo4j db from file: path/nspos-neo4j_backup_timestamp.tar.gz? Press return to continue, or Ctrl+C to abort:
Press ↵.

The restore operation begins, and messages like the following are displayed:

TASK [dbrestore : Copy backupset] ***************************

changed: [server_IP]

TASK [dbrestore : Running nspdctl stop] *********************

changed: [server_IP]

TASK [dbrestore : Ensure database service is stopped] *******

changed: [server_IP]

TASK [dbrestore : Perform database restore] *****************

changed: [server_IP]

TASK [dbrestore : Delete temporary directory] ***************

changed: [server_IP]

PLAY RECAP **************************************************

server_IP : ok=n changed=n unreachable=n failed=n
If the failed value is greater than zero, a restore failure has occurred; contact technical support for assistance.

Restore the PostgreSQL database.

Enter the following:

# ./db-restore.sh --target IP_address ↵

where IP_address is the main server IP address

The following message and prompt are displayed:

Verifying prerequisites...

Starting database restore ...

Backupset file to restore (.tar.gz format):
Enter the following and press ↵:

path/nspos-postgresql_backup_timestamp.tar.gz

where

path is the absolute path of the PostgreSQL backup file

timestamp is the backup creation time

Note: PostgreSQL backup files are stored in the following locations on a main server, depending on the backup type:
- scheduled backup—/opt/nsp/os/backup/backupset_n
- manual backup—/opt/nsp/os/backup/manual_timestamp
The following messages and prompt are displayed:

PLAY [all] **************************************************

[dbrestore : pause]

Do you want to restore the nspOS PostgreSQL db from file: path/nspos-postgresql_backup_timestamp.tar.gz? Press return to continue, or Ctrl+C to abort:
Press ↵.

The restore operation begins, and messages like the following are displayed:

TASK [dbrestore : Running nspdctl stop] *********************

changed: [server_IP]

TASK [dbrestore : Perform database restore] *****************

changed: [server_IP]

TASK [dbrestore : Delete temporary directory] ***************

changed: [server_IP]

PLAY RECAP **************************************************

server_IP : ok=n changed=n unreachable=n failed=n
If the failed value is greater than zero, a restore failure has occurred; contact technical support for assistance.

Enable Windows Active Directory access

If you intend to use Windows Active Directory, or AD, for single-sign-on client access, you must configure LDAP remote authentication for AD; otherwise,go to Step 75.

Open the following file as a reference for use in subsequent steps:

/opt/nsp/os/install/examples/config.yml

Note: Consider the following.

The NFM-P does not assign a default user group to users of a remote authentication source that you define for Windows AD; the authentication source must provide the user group attributes.
Windows AD supports the following LDAP server types for remote authentication:

AD—The user group of an AD user is derived from the group_base_dn attribute in the server configuration; group search filters are not supported.

AUTHENTICATED—The server configuration must include bind credentials; group search filters are supported. After NFM-P initialization, you add the AD server bind credentials to the NSP password vault using the NSP Session Manager REST API.

Locate the section that begins with the following lines:

# ldap:

# enabled: true

# servers:

# - type: AUTHENTICATED/AD/ANONYMOUS

# url: ldaps://ldap.example.com:636

# security: SSL/STARTTLS/NONE

Open the following file using a plain-text editor such as vi:

/opt/nsp/os/install/config.json

Locate the section that begins with the following line:

"sso": {

The section has one subsection for each type of SSO access.

Note: You can enable multiple remote authentication methods such as LDAP and RADIUS in the config.json file, or by using the NFM-P GUI. Using the GUI also allows you to specify the order in which the methods are tried during login attempts; however, no ordering is applied to multiple methods enabled in the config.json file.

In the sso section, create an ldap subsection as shown below using the parameter names from the ldap section of config.yml and the required values for your configuration.

The following example shows the LDAP configuration for two AD servers:

    "ldap": {

      "enabled": true,

      "servers": [

          "type": "auth_type",

          "url": "ldaps://server1:port",

          "server1_parameter_1": "value",

          "server1_parameter_2": "value",

          "server1_parameter_n": "value",

},

          "type": "auth_type",

          "url": "ldaps://server2:port",

          "server2_parameter_1": "value",

          "server2_parameter_2": "value",

          "server2_parameter_n": "value",

},

}]

where auth_type is AD or AUTHENTICATED

Save and close the files.

Enter the following:

# samconfig -m main ↵

The following is displayed:

Start processing command line inputs...

<main>

Enter the following:

<main> apply ↵

The AD LDAP configuration is applied.

Enter the following:

<main> exit ↵

The samconfig utility closes.

Enable CAC access

If you do not intend to enable Common Access Card, or CAC, technology for NFM-P client access, go to Step 75.

Download the federationmetadata.xml from the following ADFS link:

https://ADFS_server_name/FederationMetadata/2007-06/federationmetadata.xml

where ADFS_server_name is the ADFS server FQDN

Add an ADFS server entry to the /etc/hosts file on the main server.

Open the /etc/hosts file using a plain-text editor such as vi.
Add the following line below the line that contains the main server IP address:

IP_address FQDN

where

IP_address is the IP address of the ADFS server

FQDN is the FQDN of the ADFS server
Save and close the file.

In order to enable CAC for client access, you must configure Active Directory Federation Services, or ADFS.

Open the following file using a plain-text editor such as vi:

/opt/nsp/os/install/config.json

In the sso section, create an saml2 subsection as shown below using the parameter names from the saml2 section of config.yml and the required values for your configuration.

The following example shows the ADFS configuration.

Note: You must preserve the lead spacing of each line.

"sso" : {

"saml2": {

"enabled": true,

"service_provider_entity_id": "NFM-P_identifier",

"service_provider_metadata_filename": "casmetadata.xml",

"maximum_authentication_lifetime": 3600,

"accepted_skew": 300,

"destination_binding": "urn:oasis:names:tc:SAML:2.0:bindings:HTTP-Redirect",

"identity_provider_metadata_path": "ADFS_metadata_file",

"authn_context_class_ref": "urn:oasis:names:tc:SAML:2.0:ac:classes:TLSClient",

"authn_context_comparison_type": "minimum",

"name_id_policy_format": "urn:oasis:names:tc:SAML:1.1:nameid-format:unspecified",

"force_auth": true,

"passive": false,

"wants_assertions_signed": false,

"wants_responses_signed": false,

"all_signature_validation_disabled": false,

"sign_service_provider_metadata": false,

"principal_id_attribute": "UPN",

"use_name_qualifier": false,

"provider_name": "ADFS_server_URI",

"requested_attributes": [{

"name": "http://schemas.xmlsoap.org/ws/2005/05/identity/claims/emailaddress",

"friendly_name": "E-Mail Address",

"name_format": "urn:oasis:names:tc:SAML:2.0:attrname-format:uri",

"required": false

} ],

"mapped_attributes": [{

"name": "http://schemas.xmlsoap.org/claims/Group",

"mapped_to": "authorizationProfile"

}, {

"name": "http://schemas.xmlsoap.org/ws/2005/05/identity/claims/upn",

"mapped_to": "upn"

} ]

},

Configure the following parameters; leave all other parameters at the default:

"service_provider_entity_id": "NFM-P_identifier"
"identity_provider_metadata_path": "ADFS_metadata_file"
"provider_name": "ADFS_server_name"

NFM-P_identifier is the unique ADFS Relying Trust Party identifier

ADFS_metadata_file is the absolute path of the ADFS metadata XML file, for example, /opt/federationmetadata.xml

ADFS_server_name is the ADFS server FQDN

Save and close the files.

Enter the following:

# samconfig -m main ↵

The following is displayed:

Start processing command line inputs...

<main>

Enter the following:

<main> apply ↵

The ADFS configuration is applied.

Enter the following:

<main> exit ↵

The samconfig utility closes.

Configure WS-NOC integration

If the NFM-P is integrated with a WS-NOC system, open the following file with a plain-text editor such as vi:

/opt/nsp/os/install/examples/config.json

Otherwise, go to Step 85.

Copy the following section:

"nfmt": {

"primary_ip": "",

"standby_ip": "",

"username": "",

"password": "",

"cert_provided": false

},

Close the file.

Open the following file with a plain-text editor such as vi:

/opt/nsp/os/install/config.json

Paste in the copied section.

Configure the required parameters to enable the WS-NOC integration:

primary_ip—the primary WS-NOC server IP address
standby_ip—the standby WS-NOC server IP address
username—the username required for WS-NOC access
password—the password required for WS-NOC access
cert_provided—whether a TLS certificate is used

Save and close the file.

Enter the following:

# samconfig -m main ↵

The following is displayed:

Start processing command line inputs...

<main>

Enter the following:

<main> apply ↵

The configuration is applied.

Enter the following:

<main> exit ↵

The samconfig utility closes.

Stop NSP analytics servers, NSP Flow Collectors

If the system includes one or more NSP analytics servers, stop each analytics server.

Log in to the analytics server station as the nsp user.
Open a console window.
Enter the following:

bash$ /opt/nsp/analytics/bin/AnalyticsAdmin.sh stop ↵

The following is displayed:

Stopping Analytics Application

When the analytics server is completely stopped, the following message is displayed:

Analytics Application is not running

If the system includes one or more NSP Flow Collector Controllers and Flow Collectors, stop each NSP Flow Collector Controller.

Note: If the NSP Flow Collector Controller is collocated on a station with an NSP Flow Collector, stopping the NSP Flow Collector Controller also stops the Flow Collector.

Log in to the NSP Flow Collector Controller station as the nsp user.
Open a console window.
Enter the following:

bash$ /opt/nsp/flow/fcc/bin/flowCollectorController.bash stop ↵

The NSP Flow Collector Controller stops.

If the system includes one or more NSP Flow Collectors that are not collocated on a station with a Flow Collector Controller, stop each such NSP Flow Collector.

Log in to the NSP Flow Collector station as the nsp user.
Open a console window.
Enter the following:

bash$ /opt/nsp/flow/fc/bin/flowCollector.bash stop ↵

The NSP Flow Collector stops.

Upgrade auxiliary servers [Aux2]

If the system includes auxiliary servers, perform To upgrade a Release 22.9 or later NFM-P auxiliary server on each appropriate auxiliary server station [Aux2].

Verify auxiliary database synchronization

If the system does not include redundant auxiliary database clusters, go to Step 94.

If you are upgrading the first redundant auxiliary database cluster, you must verify the success of the most recent copy-cluster operation, which synchronizes the database data between the redundant clusters.

Note: You must not proceed to the next step until the copy-cluster operation is complete and successful.

Perform one of the following periodically to check the copy-cluster status.

If the NFM-P is in a shared-mode NSP deployment, issue the following REST API call:

Note: In order to issue a REST API call, you require a REST token; see this tutorial on the Network Developer Portal for information.

GET https://address:8545/restconf/data/auxdb:/auxdb-agent

where address is the advertised address of the primary NSP cluster

The call returns a status of SUCCESS, as shown below, for a successfully completed copy-cluster operation:

<HashMap>

      <auxdb-agent>

         <name>nspos-auxdb-agent</name>

         <application-mode>ACTIVE</application-mode>

         <copy-cluster>

            <source-cluster>cluster_M</source-cluster>

            <target-cluster>cluster_N</target-cluster>

            <time-started>timestamp</time-started>

            <status>SUCCESS</status>

         </copy-cluster>

      </auxdb-agent>

</HashMap>
If the NFM-P is not in a shared-mode NSP deployment, enter the following as the root user on the primary main server station [Main1]:

# /opt/nsp/os/nspd/nspdctl auxdb agent-status ↵

The command returns output like the following for a successfully completed copy-cluster operation:

     DC-ROLE HOST APPLICATION-MODE

     active leader 203.0.113.101 ACTIVE

     Copy Cluster Details

     SOURCE TARGET TIME-STARTED STATUS

     cluster_1 cluster_2 2022-03-14T15:09:26.535Z SUCCESS

Enable maintenance mode on auxiliary database agent

Perform one of the following to enable nspos-auxdb-agent maintenance mode.

If the NFM-P is in a shared-mode NSP deployment, perform the following steps.
1. Log in as the root user on the NSP cluster host in the primary data center.
2. Enter the following to set the nspos-auxdb-agent mode to maintenance:
  
  # kubectl patch configmap/nspos-auxdb-agent-overrides --type=merge -p '{"data":{"nspos-auxdb-agent-overrides.json":"{\"auxDbAgent\":{\"config\":{\"maintenance-mode\":true}}}"}}' ↵
3. Enter the following to restart the nspos-auxdb-agent pod:
  
  # kubectl delete pod `kubectl describe pods | grep -P ^^Name: | grep -oP nspos-auxdb-agent[-a-zA-Z0-9]+` ↵
4. Issue the following REST API call against the primary NSP cluster to verify that the agent is in maintenance mode:
  
  NOTE: In order to issue a REST API call, you require a REST token; see this tutorial on the Network Developer Portal for information.
  
  GET https://address:8545/restconf/data/auxdb:/auxdb-agent
  
  where address is the advertised address of the primary NSP cluster
  
  The call returns information like the following:
  
  {
  
      "auxdb-agent": {
  
          "name": "nspos-auxdb-agent",
  
          "application-mode": "MAINTENANCE",
  
          "copy-cluster": {
  
              "source-cluster": "cluster_2",
  
              "target-cluster": "cluster_1",
  
              "time-started": "timestamp",
  
              "status": "SUCCESS"
  
          }
  
      }
  
  }
  
  The agent is in maintenance mode if the application-mode is MAINTENANCE, as shown in the example.
5. Log in as the root user on the NSP cluster host in the standby data center.
6. Enter the following to set the nspos-auxdb-agent mode to maintenance:
  
  # kubectl patch configmap/nspos-auxdb-agent-overrides --type=merge -p '{"data":{"nspos-auxdb-agent-overrides.json":"{\"auxDbAgent\":{\"config\":{\"maintenance-mode\":true}}}"}}' ↵
If the NFM-P is not in a shared-mode NSP deployment, perform the following steps.
1. Log in as the root user on the NSP cluster host in the primary data center.
2. Enter the following to set the nspos-auxdb-agent mode to maintenance:
  
  # sed -i -r 's/("maintenance-mode"\s*:\s*)false/\1true/g' /opt/nsp/os/auxdb-agent/conf/nspos-auxdb-agent-overrides.json ↵
3. Enter the following to verify that the nspos-auxdb-agent is in maintenance mode:
  
  # /opt/nsp/os/nspd/nspdctl auxdb agent-status ↵
  
  DC-ROLE         HOST             APPLICATION-MODE
  
  active leader   203.0.113.101    MAINTENANCE
  
  standby leader  203.0.113.102    inactive
  
  The agent is in maintenance mode if the APPLICATION-MODE of the active leader is MAINTENANCE, as shown in the example.
4. Log in as the root user on the NSP cluster host in the standby data center.
5. Enter the following to set the nspos-auxdb-agent mode to maintenance:
  
  # sed -i -r 's/("maintenance-mode"\s*:\s*)false/\1true/g' /opt/nsp/os/auxdb-agent/conf/nspos-auxdb-agent-overrides.json ↵

Upgrade standby auxiliary database cluster

If you are upgrading the first redundant auxiliary database cluster, perform the following steps to stop the database proxy on each station in each auxiliary database cluster.

Enter the following sequence of commands as the root user on each auxiliary database station in the standby data center:

# systemctl stop nfmp-auxdbproxy.service ↵

# systemctl disable nfmp-auxdbproxy.service ↵

The proxy stops, and is disabled.
Enter the following sequence of commands as the root user on each auxiliary database station in the primary data center:

# systemctl stop nfmp-auxdbproxy.service ↵

# systemctl disable nfmp-auxdbproxy.service ↵

The proxy stops, and is disabled.

Perform To upgrade a Release 22.9 or later NFM-P auxiliary database cluster to upgrade the standby auxiliary database cluster.

Stop and disable original primary main server [Main1]

Stop the original primary main server.

Note: This step marks the beginning of the network management outage.

Log in to the original primary main server station [Main1] as the nsp user.
Open a console window.
Enter the following:

bash$ cd /opt/nsp/nfmp/server/nms/bin ↵
Enter the following:

bash$ ./nmsserver.bash stop ↵
Enter the following:

bash$ ./nmsserver.bash appserver_status ↵

The server status is displayed; the server is fully stopped if the status is the following:

Application Server is stopped

If the server is not fully stopped, wait five minutes and then repeat this step. Do not perform the next step until the server is fully stopped.
Enter the following to switch to the root user:

bash$ su ↵
If the NFM-P is not part of a shared-mode NSP deployment, enter the following to display the nspOS service status:

# nspdctl status ↵

Information like the following is displayed.

Mode:     DR

Role:     redundancy_role

DC-Role:  dc_role

DC-Name:  dc_name

Registry: IP_address:port

State:    stopped

Uptime:   0s

SERVICE           STATUS

service_a         inactive

service_b         inactive

service_c         inactive

You must not proceed to the next step until all NSP services are stopped; if the State is not ‘stopped’, or the STATUS indicator of each listed service is not ‘inactive’, repeat this substep.

Disable the automatic main server startup so that the main server does not start in the event of a power disruption during the upgrade.

Enter the following:

# systemctl disable nspos-nspd.service ↵
Enter the following:

# systemctl disable nfmp-main-config.service ↵
Enter the following:

# systemctl disable nfmp-main.service ↵

Upgrade NSP Flow Collector Controllers, Flow Collectors

If the system includes one or more NSP Flow Collectors, upgrade each NSP Flow Collector Controller and Flow Collector as described in NSP Flow Collector and Flow Collector Controller upgrade from Release 22.9 or later.

Stop auxiliary servers [Aux1]

If the system includes auxiliary servers, perform the following steps on each [Aux1] auxiliary server station.

Log in to the auxiliary server station as the nsp user.
Open a console window.
Enter the following:

bash$ /opt/nsp/nfmp/auxserver/nms/bin/auxnmsserver.bash auxstop ↵

The auxiliary server stops.

Stop original primary main database [DB1]

Open a console window.

100

Stop and disable the Oracle proxy and main database services.

Enter the following to stop the Oracle proxy:

# systemctl stop nfmp-oracle-proxy.service ↵
Enter the following to disable the automatic Oracle proxy startup:

# systemctl disable nfmp-oracle-proxy.service ↵
Enter the following to stop the main database:

# systemctl stop nfmp-main-db.service ↵
Enter the following to disable the automatic database startup:

# systemctl disable nfmp-main-db.service ↵

101

Perform the following steps.

Perform To apply a RHEL update to an NSP image-based OSon the main database station.
Open the /etc/fstab file using a plain-text editor such as vi.
Locate the tmpfs file system entry.
Remove the noexec option so that the entry reads as follows:

tmpfs /dev/shm tmpfs nodev 0 0
Save and close the /etc/fstab file.
Enter the following to remount the /dev/shm partition:

# mount -o remount /dev/shm ↵

Upgrade auxiliary database, if not redundant

102

If the system does not include an auxiliary database, go to Step 106.

103

If the system includes a standalone auxiliary database, perform the following steps.

Perform To upgrade a Release 22.9 or later NFM-P auxiliary database cluster.
Go to Step 106.

Enable maintenance mode for auxiliary database agent

104

If the system includes redundant auxiliary database clusters, and the NFM-P is not in a shared-mode NSP deployment, enter the following as the root user on the newly upgraded main server [Main2]:

# sed -i -r 's/("maintenance-mode"\s*:\s*)false/\1true/g' /opt/nsp/os/auxdb-agent/conf/nspos-auxdb-agent-overrides.json ↵

The auxiliary database cluster enters maintenance mode within approximately one minute.

Stop former primary auxiliary database cluster

105

If the system includes redundant auxiliary database clusters, perform the following steps on one station in the upgraded former primary cluster.

Log in as the root user.
Open a console window.
Enter the following:

# cd /opt/nsp/nfmp/auxdb/install/bin ↵
Enter the following to stop the auxiliary database:

# ./auxdbAdmin.sh stop ↵
Enter the following to display the auxiliary database status:

# ./auxdbAdmin.sh status ↵

Information like the following is displayed:

Database status

Node | Host | State | Version | DB

------------+---------------+-------+---------+-------

node_1 | internal_IP_1 | STATE | version | db_name

node_2 | internal_IP_2 | STATE | version | db_name

.

.

.

node_n | internal_IP_n | STATE | version | db_name

Output captured in log_file

The cluster is stopped when each STATE entry reads DOWN.
Repeat substep 5 periodically until the cluster is stopped.

Note: You must not proceed to the next step until the cluster is stopped.

Start new primary main server [Main2]

106

CAUTION

Service Disruption

The new primary database [DB2] must be upgraded and running before you start the new primary main server [Main2], or the main server initialization may fail.

If you perform the new primary main server and database upgrades concurrently, do not perform this step until the database upgrade is complete.

CAUTION

Service Disruption

An NFM-P system upgrade is not complete until each main server performs crucial post-upgrade tasks during initialization.

Before you attempt an operation that requires a server shutdown, you must ensure that each main server is completely initialized, or the operation fails.

Start the new primary main server [Main2].

Note: You cannot start a main server unless the main server configuration includes a current and valid license. You can use samconfig to specify the license file, or import a license, as described in the NSP System Administrator Guide.

Log in as the nsp user on the new primary main server station [Main2].
Enter the following:

bash$ cd /opt/nsp/nfmp/server/nms/bin ↵
Enter the following:

bash$ ./nmsserver.bash start ↵
Enter the following:

bash$ ./nmsserver.bash appserver_status ↵

The server status is displayed; the server is fully initialized if the status is the following:

Application Server process is running. See nms_status for more detail.

If the server is not fully initialized, wait five minutes and then repeat this step. Do not perform the next step until the server is fully initialized.

Note: This marks the end of the network management outage.

107

If you have enabled CAC for NFM-P client access, download the casmetadata.xml file from the following URL, and then import the file into the ADFS server relying-trust-party:

https://server/cas/sp/metadata

where server is the main server IP address or hostname

After the download, the casmetadata.xml file is available in the following directory on the main server:

/opt/nsp/os/tomcat/conf/cas/saml

108

If you have enabled Windows Active Directory access using the AUTHENTICATED type of LDAP server, perform the following steps.

Use the NSP Session Manager REST API to add the LDAP server bind credentials; see the Network Developer Portal for information.
If the NFM-P is not part of a shared-mode NSP deployment, enter the following to restart the local nspos-tomcat service:

Note: The service restart may take a few minutes, during which NFM-P GUI and REST client access is degraded. General NFM-P operation is unaffected.

# systemctl restart nspos-tomcat ↵

109

Specify the memory requirement for GUI clients based on the type of network that the NFM-P is to manage.

Enter the following:

bash$ ./nmsdeploytool.bash clientmem -option ↵

where option is one of the following:
- m—medium, for management of limited-scale network
- l—large, for a network of 15 000 or more NEs
Record the setting, which is not preserved through an upgrade, for future use.
Enter the following to commit the configuration change:

bash$ ./nmsdeploytool.bash deploy ↵

Start auxiliary servers [Aux2]

110

If the NFM-P system includes auxiliary servers, start each appropriate auxiliary server [Aux2].

Log in to the auxiliary server station as the nsp user.
Open a console window.
Enter the following:

bash$ cd /opt/nsp/nfmp/auxserver/nms/bin ↵
Enter the following:

bash$ ./auxnmsserver.bash auxstart ↵

The auxiliary server starts.

Activate upgraded former standby auxiliary database cluster

111

If the system does not include redundant auxiliary database clusters, go to Step 114.

112

Perform the following steps on each station in the upgraded former standby auxiliary database cluster.

Log in as the root user on the station.
Open a console window.
Enter the following sequence of commands to enable the database services:

# systemctl enable nspos-auxdb.service ↵

# systemctl enable nspos-auxdbproxy.service ↵

# systemctl enable vertica_agent.service ↵

# systemctl enable verticad.service ↵

The services are enabled.
Enter the following to start the database proxy:

# systemctl start nspos-auxdbproxy.service ↵

The proxy starts.

113

Perform one of the following to activate the former standby auxiliary database cluster, after which the cluster assumes the primary role.

If the NFM-P is in a shared-mode NSP deployment, issue the following REST API call:

Note: In order to issue a REST API call, you require a REST token; see this tutorial on the Network Developer Portal for information.

POST https://{{address}}:8545/restconf/data/auxdb:/clusters/cluster=cluster_N/activate

where

address is the advertised address of the primary NSP cluster

N is the auxiliary database cluster number

The following is the request body:

{

  "auxdb:input" : {

    "force": true

  }

}

The cluster is activated.
If the NFM-P is not in a shared-mode NSP deployment, enter the following as the root user on the new primary main server station:

# nspdctl auxdb activate cluster_N --force ↵

where N is the auxiliary database

A message like the following is displayed:

Auxiliary database activation request submitted for [cluster_N]

Upgrade analytics servers

114

If the system includes one or more NSP analytics servers, upgrade each analytics server as described in NSP analytics server upgrade from Release 22.9 or later.

Enable GUI client

115

You require an NFM-P GUI client to complete the procedure; see the following for information:

Note: A client delegate server installation typically takes more time than the other options. A single-user client or client delegate server upgrade is recommended if your maintenance period is limited.

Test upgraded system using GUI client

116

When the new primary main server [Main2] is started, use a newly installed or upgraded GUI client to perform sanity testing of the new primary main server and database.

Note: To back out of the upgrade and return the original primary main server [Main1] and database [DB1] to service, you can do so by stopping the new primary main server [Main2] and database [DB2] and restarting the original primary main server [Main1] and database [DB1].

Uninstall original primary database [DB1]

117

Enter the following to uninstall the original primary main database:

# dnf remove nsp-nfmp-main-db nsp-nfmp-oracle ↵

The dnf utility resolves any dependencies and displays the following prompt:

Installed size: nn G

Is this ok [y/N]:

118

Enter y. The following is displayed as the packages are removed:

Downloading Packages:

Running transaction check

Transaction check succeeded.

Running transaction test

Transaction test succeeded.

Running transaction check

Uninstalling the NFM-P package...

As each package removal completes, the following is displayed:

Complete!

Install new standby main database [DB1]

119

Note: After the upgrade, the station is the new standby main database station.

120

Perform one of the following.

If the main server and database are collocated on one station, perform the following steps.
1. Transfer the following downloaded installation files to an empty directory on the collocated station:
  - nsp-nfmp-oracle-R.r.p-rel.v.rpm
  - nsp-nfmp-main-db-R.r.p-rel.v.rpm
  - nsp-nfmp-nspos-R.r.p.rpm
  - nsp-nfmp-jre-R.r.p-rel.v.rpm
  - nsp-nfmp-config-R.r.p-rel.v.rpm
  - nsp-nfmp-main-server-R.r.p.rpm
  Note: In subsequent steps, the directory is called the NFM-P software directory.
2. You must remove the semvalidator package if it is installed; otherwise, the upgrade is blocked.
  
  Enter the following:
  
  # rpm -q nsp-nfmp-semvalidator ↵
  
  If the package is installed, the following is displayed:
  
  nsp-nfmp-semvalidator-version
  
  If the package is not installed, the following is displayed:
  
  package nsp-nfmp-semvalidator is not installed
3. If the package is installed, enter the following:
  
  # dnf remove nsp-nfmp-semvalidator ↵
  
  The package is removed.
If the main server and database are on separate stations, transfer the following downloaded installation files to an empty directory on the main database station:
- nsp-nfmp-jre-R.r.p-rel.v.rpm
- nsp-nfmp-config-R.r.p-rel.v.rpm
- nsp-nfmp-oracle-R.r.p-rel.v.rpm
- nsp-nfmp-main-db-R.r.p-rel.v.rpm
- nsp-nfmp-nodeexporter-R.r.p-rel.v.rpm, if downloaded
Note: In subsequent steps, the directory is called the NFM-P software directory.

121

Transfer the following downloaded file to an empty directory on the main database station:

OracleSw_PreInstall.sh

122

Open a console window.

123

Navigate to the directory that contains the OracleSw_PreInstall.sh file.

124

Enter the following:

# chmod +x OracleSw_PreInstall.sh ↵

125

Enter the following:

# ./OracleSw_PreInstall.sh ↵

Note: A default value is displayed in brackets []. To accept the default, press ↵.

The following prompt is displayed:

This script will prepare the system for a new install/restore of an NFM-P Version R.r Rn database.

Do you want to continue? [Yes/No]:

126

Enter Yes. The following prompt is displayed:

Enter the Oracle dba group name [group]:

127

Press ↵ to accept the default.

The following messages and prompt are displayed:

Creating group group if it does not exist...

WARNING: Group group already exists locally.

Do you want to use the existing group? [Yes/No]:

128

Enter Yes.

The following message and prompt are displayed:

The user [username] for the group [group] already exists locally.

Do you want to use the existing user? [Yes/No]:

129

Enter Yes.

The following messages and prompt are displayed:

Checking or Creating the Oracle user home directory /opt/nsp/nfmp/oracle19...

Checking user username...

WARNING: Oracle user with the specified name already exists locally.

Redefining the primary group and home directory of user username ... usermod: no changes

Changing ownership of the directory /opt/nsp/nfmp/oracle19 to username:group.

About to unlock the UNIX user [username]

Unlocking password for user username.

passwd: Success

Unlocking the UNIX user [username] completed

Do you want to change the password for the user username? [Yes/No]:

130

Perform one of the following.

If you did not change the password during the upgrade of the original standby database, enter No.
If you changed the password during the upgrade of the original standby database, perform the following steps.
1. Enter Yes. The following prompt is displayed:
  
  New Password:
2. Enter a password. The following prompt is displayed:
  
  Re-enter new Password:
3. Re-enter the password. The following is displayed if the password change is successful:
  
  passwd: password successfully changed for user

The following message and prompt are displayed:

Specify whether an NFM-P nserver will be installed on this workstation.

The database memory requirements will be adjusted to account for the additional load.

Will the database co-exist with an NFM-P server on this workstation [Yes/No]:

131