To upgrade a redundant Release 22.6 or earlier NFM-P system

Description

The following steps describe how to upgrade a collocated or distributed Release 22.6 or earlier main database and main server in a redundant deployment. The steps include links to procedures for installing and upgrading optional NFM-P components.

Ensure that you record the information that you specify, for example, directory names, passwords, and IP addresses.

Note: You require the following user privileges:

Note: The following RHEL CLI prompts in command lines denote the active user, and are not to be included in typed commands:

Steps
Check pre-upgrade disk space
 

As part of the trial upgrade on a lab system in advance of a live upgrade, you must ensure that the available disk capacity on each NFM-P component remains within tolerance.

Note: If the disk usage on an NFM-P partition approaches or exceeds 80% after the trial upgrade, you may need to add disk capacity before you attempt the upgrade on a live system.

Perform the following steps on each of the following stations:

  • main server

  • auxiliary server

  • main database

  • auxiliary database

  1. Log in to the station as the root user.

  2. Open a console window.

  3. Enter the following:

    df -kh ↵

    The usage information for each partition is displayed.

  4. Record the information for each NFM-P partition; see the tables in Chapter 2, NSP disk setup and partitioning for the partition names and required capacities.


Stop and disable standby main server [Main2]
 

Open a GUI client to monitor the network during the upgrade.


Stop the standby main server.

  1. Log in to the standby main server station as the nsp user.

  2. Open a console window.

  3. Enter the following:

    bash$ cd /opt/nsp/nfmp/server/nms/bin ↵

  4. Enter the following:

    bash$ ./nmsserver.bash stop ↵

  5. Enter the following:

    bash$ ./nmsserver.bash appserver_status ↵

    The server status is displayed; the server is fully stopped if the status is the following:

    Application Server is stopped

    If the server is not fully stopped, wait five minutes and then repeat this step. Do not perform the next step until the server is fully stopped.

  6. Enter the following to switch to the root user:

    bash$ su ↵

  7. If the NFM-P is not part of a shared-mode NSP deployment, enter the following to display the nspOS service status:

    nspdctl status ↵

    Information like the following is displayed.

    Mode:     DR

    Role:     redundancy_role

    DC-Role:  dc_role

    DC-Name:  dc_name

    Registry: IP_address:port

    State:    stopped

    Uptime:   0s

    SERVICE           STATUS

    service_a         inactive

    service_b         inactive

    service_c         inactive

    You must not proceed to the next step until all NSP services are stopped; if the State is not ‘stopped’, or the STATUS indicator of each listed service is not ‘inactive’, repeat this substep.


Disable the automatic main server startup so that the main server does not start in the event of a power disruption during the upgrade.

  1. Enter the following:

    systemctl disable nspos-nspd.service ↵

  2. Enter the following:

    systemctl disable nfmp-main-config.service ↵

  3. Enter the following:

    systemctl disable nfmp-main.service ↵


Stop auxiliary servers [Aux2]
 

If the NFM-P system includes auxiliary servers, stop each appropriate auxiliary server [Aux2].

  1. Log in to the auxiliary server station as the nsp user.

  2. Open a console window.

  3. Enter the following:

    bash$ /opt/nsp/nfmp/auxserver/nms/bin/auxnmsserver.bash auxstop ↵

    The auxiliary server stops.


Disable database redundancy
 

Disable the main database failover and switchover functions.

  1. Log in to the primary main server station [Main1] as the nsp user.

  2. Open a console window.

  3. Enter the following to navigate to the main server configuration directory:

    bash$ cd /opt/nsp/nfmp/server/nms/config ↵

  4. Make a backup copy of the nms-server.xml file.

  5. Open the nms-server.xml file with a plain-text editor, for example, vi.

  6. Locate the section that begins with the following tag:

    <db

  7. Locate the following line in the section:

    host="address"

  8. Ensure that the address value in the line is the IP address of main database [DB1].

  9. Locate the following line in the section:

    database="instance_name"

  10. Ensure that the instance_name value is the instance name of main database [DB1].

  11. Edit the following line in the section that reads:

    redundancyEnabled="true"

    to read:

    redundancyEnabled="false"

  12. Save and close the nms-server.xml file.

  13. Enter the following:

    bash$ /opt/nsp/nfmp/server/nms/bin/nmsserver.bash read_config ↵

    The main server puts the change into effect, and database redundancy is disabled.


Upgrade standby main database [DB2]
 

Log in to the standby main database [DB2] station as the root user.

Note: After the upgrade, the station is the new primary main database station.


Open a console window.


Stop and disable the Oracle proxy and main database services.

  1. Enter the following to stop the Oracle proxy:

    systemctl stop nfmp-oracle-proxy.service ↵

  2. Enter the following to disable the automatic Oracle proxy startup:

    systemctl disable nfmp-oracle-proxy.service ↵

  3. Enter the following to stop the main database:

    systemctl stop nfmp-main-db.service ↵

  4. Enter the following to disable the automatic database startup:

    systemctl disable nfmp-main-db.service ↵


10 

If analytics aggregations are enabled, perform the following steps to disable all aggregation rules.

Note: Disabling analytics aggregation during a redundant system upgrade prevents the duplication of aggregation data in the NFM-P database, but does not cause the loss of any aggregation data.

Upon startup, if a primary main server detects that the most recent aggregation data is not current, the server performs the interim aggregations. If aggregation is enabled during a redundant upgrade, the original primary main server creates aggregations while the standby main server is upgraded. In such a case, after the standby main server starts as the new primary main server, the server may perform aggregations that are duplicates of the aggregations performed by the original primary main server.

The required aggregation rules are automatically enabled on the new primary main server, so the server performs the interim aggregations upon startup. If aggregation is disabled at the start of a redundant upgrade, no aggregation duplication occurs.

  1. Open an NFM-P GUI client.

  2. Choose Tools→Analytics→Aggregation Manager from the NFM-P main menu. The Aggregation Manager form opens.

  3. Click Search. The aggregation rules are listed.

  4. Click on the Enable Aggregation column to sort the rules so that the rules that have aggregation enabled are at the top of the list.

  5. Select all rules that have a check mark in the Enable Aggregation column.

  6. Click Properties. The Aggregation Rule (multiple instances) [Edit] form opens.

  7. Deselect Enable Aggregation.

  8. Click OK. The Aggregation Rule (multiple instances) [Edit] form closes.

  9. Click OK to save your changes and close the Aggregation Manager form.

  10. Close the NFM-P GUI client.


11 

Perform the following steps.

  1. Open the /etc/fstab file using a plain-text editor such as vi.

  2. Locate the tmpfs file system entry.

  3. Remove the noexec option so that the entry reads as follows:

    tmpfs /dev/shm tmpfs nodev 0 0

  4. Save and close the /etc/fstab file.

  5. Enter the following to remount the /dev/shm partition:

    mount -o remount /dev/shm ↵


12 

If you are re-using the standby main database [DB2] station, recommission the station according to the platform specifications in this guide and in the NSP Planning Guide.

For information about deploying the RHEL OS using an NSP OEM disk image, see NSP disk-image deployment.

Note: After the upgrade, the station is the new primary main database station.


13 

Log in as the root user on the station that is commissioned as the main database [DB2] station.


14 

Perform one of the following.

  1. If the main server and database are collocated on one station, perform the following steps.

    1. Enter the following:

      mkdir /opt/importConfigs ↵

    2. Transfer the mainserverBackupConfigs.tar.gz file created in Step 32 of To prepare for an NFM-P system upgrade from Release 22.6 or earlier to the /opt/importConfigs directory.

    3. Transfer the following downloaded installation files to an empty directory on the collocated station:

      • nsp-nfmp-oracle-R.r.p-rel.v.rpm

      • nsp-nfmp-main-db-R.r.p-rel.v.rpm

      • nsp-nfmp-nspos-R.r.p.rpm

      • nsp-nfmp-jre-R.r.p-rel.v.rpm

      • nsp-nfmp-config-R.r.p-rel.v.rpm

      • nsp-nfmp-main-server-R.r.p.rpm

      Note: In subsequent steps, the directory is called the NFM-P software directory.

  2. If the main server and database are on separate stations, transfer the following downloaded installation files to an empty directory on the main database station:

    • nsp-nfmp-jre-R.r.p-rel.v.rpm

    • nsp-nfmp-config-R.r.p-rel.v.rpm

    • nsp-nfmp-oracle-R.r.p-rel.v.rpm

    • nsp-nfmp-main-db-R.r.p-rel.v.rpm

    • nsp-nfmp-nodeexporter-R.r.p-rel.v.rpm, if downloaded

    Note: In subsequent steps, the directory is called the NFM-P software directory.


15 

Transfer the following downloaded file to an empty directory on the main database station:

  • OracleSw_PreInstall.sh


16 

Navigate to the directory that contains the OracleSw_PreInstall.sh file.


17 

Enter the following:

chmod +x OracleSw_PreInstall.sh ↵


18 

Enter the following:

./OracleSw_PreInstall.sh ↵

Note: A default value is displayed in brackets []. To accept the default, press ↵.

Note: If you specify a value other than the default, you must record the value for use when the OracleSw_PreInstall.sh script is run during a software upgrade, or when the Oracle management user information is required by technical support.

The following prompt is displayed:

This script will prepare the system for a new install/restore of an NFM-P Version Release main database.

Do you want to continue? [Yes/No]:


19 

Enter Yes. The following prompt is displayed:

Enter the Oracle dba group name [group]:


20 

Enter a group name.

Note: To reduce the complexity of subsequent software upgrades and technical support activities, it is recommended that you accept the default.

The following messages and prompt are displayed:

Creating group group if it does not exist...

done

Enter the Oracle user name:


21 

Enter a username.

Note: To reduce the complexity of subsequent software upgrades and technical support activities, it is recommended that you accept the default.

The following messages and prompt are displayed:

Oracle user [username] new home directory will be [/opt/nsp/nfmp/oracle19].

Checking or Creating the Oracle user home directory /opt/nsp/nfmp/oracle19...

Checking user username...

Adding username...

Changing ownership of the directory /opt/nsp/nfmp/oracle19 to username:group.

About to unlock the UNIX user [username]

Unlocking password for user username.

passwd: Success

Unlocking the UNIX user [username] completed

Please assign a password to the UNIX user username ..

New Password:


22 

Enter a password. The following prompt is displayed:

Re-enter new Password:


23 

Re-enter the password. The following is displayed if the password change is successful:

passwd: password successfully changed for username

The following message and prompt are displayed:

Specify whether an NFM-P Main Server will be installed on this workstation.

The database memory requirements will be adjusted to account for the additional load.

Will the database co-exist with an NFM-P Main Server on this workstation [Yes/No]:


24 

Enter Yes or No, as required.

Messages like the following are displayed as the script execution completes:

INFO: About to set kernel parameters in /etc/sysctl.conf...

INFO: Completed setting kernel parameters in /etc/sysctl.conf...

INFO: About to change the current values of the kernel parameters

INFO: Completed changing the current values of the kernel parameters

INFO: About to set ulimit parameters in /etc/security/limits.conf...

INFO: Completed setting ulimit parameters in /etc/security/limits.conf...

INFO: Completed running Oracle Pre-Install Tasks, you *MUST* reboot your box.


25 

When the script execution is complete, enter the following to reboot the main database station:

systemctl reboot ↵

The station reboots.


26 

When the reboot is complete, log in to the main database [DB2] station as the root user.


27 

Open a console window.


28 

Navigate to the NFM-P software directory.

Note: Ensure that the directory contains only the installation files.


29 

Enter the following:

chmod +x * ↵


30 

Enter the following:

dnf install *.rpm ↵

The dnf utility resolves any package dependencies, and displays the following prompt:

Total size: nn G

Installed size: nn G 

Is this ok [y/d/N]: 


31 

Enter y. The following and the installation status are displayed as each package is installed:

Downloading Packages:

Running transaction check

Transaction check succeeded.

Running transaction test

Transaction test succeeded.

Running transaction check

The package installation is complete when the following is displayed:

Complete!


32 

Transfer the database backup file set to the station.

Note: The path to the backup file set must be the same as the original backup path, which is named in the BACKUP_SUMMARY.INFO file from the backup file set; for example:

Backup Path Name:

        /opt/nsp/nfmp/dbbackup/backupset_1

Note: Ensure that the Oracle management user has full access to the directory and contents.


33 

Enter the following:

samrestoreDb path -migrate ↵

where path is the absolute path of the database backup file set

The database restore begins, and messages like the following are displayed as the restore progresses.

Restore log is /opt/nsp/nfmp/db/install/NFM-P_Main_Database.restore.yyyy.mm.dd-hh.mm.ss.stdout.txt

<date time> working..

<date time> Performing Step 1 of 7 - Initializing ..

<date time> Executing StartupDB.sql ...

<date time> Performing Step 2 of 7 - Extracting backup files .....

<date time> Performing Step 3 of 7 - Restoring archive log files ..

<date time> Performing Step 4 of 7 - Executing restore.rcv ..........

<date time> Performing Step 5 of 7 - Restoring Accounting tablespaces .......

<date time> Performing Step 6 of 7 - Opening database .....

<date time> working....

<date time> Executing ConfigRestoreDB.sql ....................

<date time> working...............

<date time> Performing Step 7 of 7 - Configuring NFM-P Server settings ...

The following is displayed when the restore is complete:

<date time> Database restore was successful

DONE


34 

Stop the Oracle proxy and main database services.

  1. Enter the following to stop the Oracle proxy:

    systemctl stop nfmp-oracle-proxy.service ↵

  2. Enter the following to stop the main database:

    systemctl stop nfmp-main-db.service ↵


35 

You must prepare the restored database for the upgrade.

Navigate to the directory that contains the OracleSw_PreInstall.sh file.


36 

Enter the following:

./OracleSw_PreInstall.sh ↵

Note: A default value is displayed in brackets []. To accept the default, press ↵.

Note: If you specify a value other than the default, you must record the value for use when the OracleSw_PreInstall.sh script is run during a software upgrade, or when the Oracle management user information is required by technical support.

The following prompt is displayed:

This script will prepare the system for an upgrade to NFM-P Version R.r Rn database.

Do you want to continue? [Yes/No]:


37 

Enter Yes. The following messages and prompt are displayed:

About to validate that the database can be upgraded to release.

Found the database installation directory /opt/nsp/nfmp/samdb/install.

Existing NFM-P database version = version

Enter the password for the "SYS" Oracle user (terminal echo is off):


38 

Enter the SYS user password.

The script begins to validate the database records, and displays the following:

Validating the database for upgrade. Please wait ...

If the validation is successful, the following messages and prompt are displayed:

INFO: Database upgrade validation passed.

Creating group group if it does not exist ...

Checking or Creating the Oracle user home directory /opt/nsp/nfmp/oracle19...

Checking user username...

usermod: no changes

Changing ownership of the directory /opt/nsp/nfmp/oracle19 to username:group.

About to unlock the UNIX user [username]

Unlocking password for user username.

passwd: Success

Unlocking the UNIX user [username] completed

Do you want to change the password for the UNIX user username? [Yes/No]:


39 

If the database contains an invalid item, for example, an NE at a release that the new NFM-P software does not support, the following is displayed and the script exits; otherwise, go to Step 40.

ERROR: Unsupported records found in database. Please remove the following unsupported items first:

Please remove the following unsupported items first:

item_1

item_2

.

.

item_n

ERROR: The database cannot be upgraded. Please fix the above errors and re-run this script.

Perform the following steps.

  1. Use an NFM-P GUI client to remove or update the unsupported items, as required. For example, upgrade an unsupported NE to a release that the new software supports.

  2. Run the script again; go to Step 36.


40 

Perform one of the following.

  1. Enter No to retain the current password.

  2. Specify a new password.

    1. Enter Yes. The following prompt is displayed:

      New Password: 

    2. Enter a password. The following prompt is displayed:

      Re-enter new Password:

    3. Re-enter the password. The following is displayed if the password change is successful:

      passwd: password successfully changed for username

The following message and prompt are displayed:

Specify whether an NFM-P server will be installed on this workstation.

The database memory requirements will be adjusted to account for the additional load.

Will the database co-exist with an NFM-P server on this workstation [Yes/No]:


41 

Enter Yes or No, as required.

Messages like the following are displayed as the script execution completes:

INFO: About to remove kernel parameters set by a previous run of this script from /etc/sysctl.conf

INFO: Completed removing kernel parameters set by a previous run of this script from /etc/sysctl.conf

INFO: About to set kernel parameters in /etc/sysctl.conf...

INFO: Completed setting kernel parameters in /etc/sysctl.conf...

INFO: About to change the current values of the kernel parameters

INFO: Completed changing the current values of the kernel parameters

INFO: About to remove ulimit parameters set by a previous run of this script from /etc/security/limits.conf

INFO: Completed removing ulimit parameters set by a previous run of this script from /etc/security/limits.conf

INFO: About to set ulimit parameters in etc/security/limits.conf...

INFO: Completed setting ulimit parameters in /etc/security/limits.conf...

INFO: Completed running Oracle Pre-Install Tasks


42 

Enter the following to upgrade the database:

Note: A database upgrade takes considerable time that varies, depending on the release from which you are upgrading.

samupgradeDb ↵

The following prompt is displayed:

Enter the password for the "SAMUSER" database user (terminal echo is off):


43 

Enter the database user password.

The database upgrade begins, and messages like the following are displayed:

Validation succeeded.

Upgrade log is /opt/nsp/nfmp/db/install/NFM-P_Main_Database.upgrade.timestamp.stdout.txt

Performing Step 1 of n - Initializing ...........

Performing NFM-P database upgrade will take time...

Executing PreOraUpgrade.sql .............

Executing ShutdownDBUpgrade.sql ....

Executing StartupDB.sql .....

Executing DbJavaReload.sql ................

The database upgrade is complete when the following is displayed:

DONE


44 

Verify the database configuration and create the database.

Note: This main database [DB1] is the new primary main database.

  1. Enter the following:

    samconfig -m db ↵

    The following is displayed:

    Start processing command line inputs...

    <db> 

  2. Enter the following:

    <db> show-detail ↵

    The database configuration is displayed.

  3. Review each parameter to ensure that the value is correct; see NFM-P samconfig utility for information about using samconfig.

  4. Configure one or more parameters, if required, and then enter back ↵.

  5. Enter the following to apply the configuration and create the database:

    <db> apply ↵

    The configuration is applied, and the database creation begins.

  6. When the database creation is complete, enter the following:

    <db> exit ↵

    The samconfig utility closes.


Upgrade standby main server [Main2]
 
45 

If the main server [Main2] and database [DB2] are on separate stations, and you are re-using the [Main2] main server station, recommission the station according to the platform specifications in this guide and in the NSP Planning Guide.

For information about deploying the RHEL OS using an NSP OEM disk image, see NSP disk-image deployment.


46 

Log in as the root user on the station that is commissioned as the [Main2] station.

Note: After the upgrade, the station is the new primary main server station.


47 

Open a console window.


48 

If the main server and database are on separate stations, perform the following steps.

  1. Enter the following:

    mkdir /opt/importConfigs ↵

  2. Transfer the mainserverBackupConfigs.tar.gz file created in Step 32 of To prepare for an NFM-P system upgrade from Release 22.6 or earlier to the /opt/importConfigs directory.


49 

Perform one of the following.

  1. If the main server and database are collocated on one station, go to Step 55.

  2. If the main server and database are on separate stations, transfer the following downloaded installation files to an empty directory on the main server station:

    • nsp-nfmp-nspos-R.r.p.rpm

    • nsp-nfmp-jre-R.r.p-rel.v.rpm

    • nsp-nfmp-config-R.r.p-rel.v.rpm

    • nsp-nfmp-main-server-R.r.p.rpm

    • nsp-nfmp-nodeexporter-R.r.p-rel.v.rpm, if downloaded

    Note: In subsequent steps, the directory is called the NFM-P software directory.


50 

Navigate to the NFM-P software directory.


51 

Enter the following:

chmod +x * ↵


52 

Enter the following:

dnf install *.rpm ↵

The dnf utility resolves any package dependencies, and displays the following prompt:

Total size: nn G

Installed size: nn G 

Is this ok [y/d/N]: 


53 

Enter y. The following and the installation status are displayed as each package is installed:

Downloading Packages:

Running transaction check

Transaction check succeeded.

Running transaction test

Transaction test succeeded.

Running transaction check

The package installation is complete when the following is displayed:

Complete!


Start PKI server
 
54 

Start the PKI server, regardless of whether you are using the automated or manual TLS configuration method; perform To configure and enable a PKI server.

Note: The PKI server is required for internal system configuration purposes.


Configure new primary main server [Main2]
 
55 

Enter the following; see NFM-P samconfig utility for information about using samconfig:

Note: Regardless of whether you intend to modify the main server configuration, you must apply the main server configuration, as described in the following steps.

samconfig -m main ↵

The following is displayed:

Start processing command line inputs...

<main>


56 

Enter the following:

<main> configure ↵

The prompt changes to <main configure>.


57 

To apply a new or updated NFM-P license, enter the following:

Note: You cannot start a main server unless the main server configuration includes a current and valid license. You can use samconfig to specify the license file in this step, or later import the license, as described in the NSP System Administrator Guide.

<main configure> license license_file

where license_file is the path and file name of the NSP license bundle


58 

Enter the following:

<main configure> database instance primary_instance back ↵

where primary_instance is the [DB1] database instance name, which is the primary Instance Name recorded in Step 43 of To prepare for an NFM-P system upgrade from Release 22.6 or earlier


59 

Enter the following:

<main configure> redundancy database instance standby_instance back ↵

where standby_instance is the [DB2] database instance name, which is the standby Instance Name recorded in Step 44 of To prepare for an NFM-P system upgrade from Release 22.6 or earlier

The prompt changes to <main configure redundancy>.


60 

Enter the following:

<main configure redundancy> back ↵

The prompt changes to <main configure>.


61 

Verify the main server configuration.

  1. Enter the following:

    <main configure> show ↵

    The main server configuration is displayed.

  2. Review each parameter to ensure that the value is correct; see NFM-P samconfig utility for information about using the samconfig utility.

  3. Configure one or more parameters, if required.

    Note: The NFM-P uses the database backup settings to initialize the database during installation only. To change the backup settings after installation, you must use the Database Manager form in the NFM-P client GUI, as described in the NSP System Administrator Guide.

  4. When you are certain that the configuration is correct, enter the following:

    <main configure> back ↵

    The prompt changes to <main>.


62 

Enter the following:

<main> apply ↵

The configuration is applied.


63 

Enter the following:

<main> exit ↵

The samconfig utility closes.


64 

If the NFM-P is part of a shared-mode NSP system and you want to enable mTLS for internal Kafka authentication using two-way TLS, perform the following steps.

Note: Enabling mTLS for internal Kafka authentication is supported only in an NSP deployment that uses separate interfaces for internal and client communication.

Note: The parameter you must configure is displayed only if the ip-list parameter is set to a remote address.

Note: The parameter is configurable only if the secure and internal-certs parameters in the nspos section are set to true.

  1. Enter the following:

    samconfig -m main ↵

    The following is displayed:

    Start processing command line inputs...

    <main> 

  2. Enter the following:

    configure nspos mtls-kafka-enabled back ↵

  3. Enter the following:

    <main> apply ↵

    The configuration is applied.

  4. Enter the following:

    <main> exit ↵

    The samconfig utility closes.


Restore embedded nspOS, independent deployment
 
65 

In an independent NFM-P deployment, you must restore the embedded Neo4j and PostgreSQL databases. Otherwise, if the NFM-P is integrated with an NSP cluster, go to Step 83.


66 

Enter the following:

mkdir /opt/nsp/os/backup ↵


67 

Enter the following:

chown nsp:nsp /opt/nsp/os/backup ↵


68 

Copy the Neo4j and PostgreSQL backup files saved in Step 31 of To prepare for an NFM-P system upgrade from Release 22.6 or earlier to the /opt/nsp/os/backup directory.


69 

Restore the Neo4j database.

  1. Enter the following:

    cd /opt/nsp/os/install/tools/database ↵

  2. Enter the following:

    ./db-restore.sh --target IP_address

    where IP_address is the main server [Main2] IP address

    The following message and prompt are displayed:

     Verifying prerequisites...

     Starting database restore ...

    Backupset file to restore (.tar.gz format):

  3. Enter the following and press ↵:

    path/nspos-neo4j_backup_timestamp.tar.gz

    where

    path is the absolute path of the Neo4j backup file

    timestamp is the backup creation time

    Note: Neo4j backup files are stored in the following locations on a main server, depending on the backup type:

    • scheduled backup—/opt/nsp/os/backup/backupset_n

    • manual backup—/opt/nsp/os/backup/manual_timestamp

    The following messages and prompt are displayed:

    PLAY [all] **************************************************

    TASK [dbrestore : Create temporary directory] ***************

    changed: [server_IP]

    [dbrestore : pause]

    Do you want to restore the nspOS Neo4j db from file: path/nspos-neo4j_backup_timestamp.tar.gz? Press return to continue, or Ctrl+C to abort:

  4. Press ↵.

    The restore operation begins, and messages like the following are displayed:

    TASK [dbrestore : Copy backupset] ***************************

    changed: [server_IP]

    TASK [dbrestore : Running nspdctl stop] *********************

    changed: [server_IP]

    TASK [dbrestore : Ensure database service is stopped] *******

    changed: [server_IP]

    TASK [dbrestore : Perform database restore] *****************

    changed: [server_IP]

    TASK [dbrestore : Delete temporary directory] ***************

    changed: [server_IP]

    PLAY RECAP **************************************************

    server_IP     : ok=n   changed=n    unreachable=n   failed=n

  5. If the failed value is greater than zero, a restore failure has occurred; contact technical support for assistance.


70 

Restore the PostgreSQL database.

  1. Enter the following:

    ./db-restore.sh --target IP_address

    where IP_address is the main server [Main2] IP address

    The following message and prompt are displayed:

     Verifying prerequisites...

     Starting database restore ...

    Backupset file to restore (.tar.gz format):

  2. Enter the following and press ↵:

    path/nspos-postgresql_backup_timestamp.tar.gz

    where

    path is the absolute path of the PostgreSQL backup file

    timestamp is the backup creation time

    Note: PostgreSQL backup files are stored in the following locations on a main server, depending on the backup type:

    • scheduled backup—/opt/nsp/os/backup/backupset_n

    • manual backup—/opt/nsp/os/backup/manual_timestamp

    The following messages and prompt are displayed:

    PLAY [all] **************************************************

    [dbrestore : pause]

    Do you want to restore the nspOS PostgreSQL db from file: path/nspos-postgresql_backup_timestamp.tar.gz? Press return to continue, or Ctrl+C to abort:

  3. Press ↵.

    The restore operation begins, and messages like the following are displayed:

    TASK [dbrestore : Running nspdctl stop] *********************

    changed: [server_IP]

    TASK [dbrestore : Perform database restore] *****************

    changed: [server_IP]

    TASK [dbrestore : Delete temporary directory] ***************

    changed: [server_IP]

    PLAY RECAP **************************************************

    server_IP     : ok=n   changed=n    unreachable=n   failed=n

  4. If the failed value is greater than zero, a restore failure has occurred; contact technical support for assistance.


Restore new primary main server [Main2] data files
 
71 

Transfer the main server data backup .tar.gz file set created in Step 34 of To prepare for an NFM-P system upgrade from Release 22.6 or earlier to the /opt/nsp/nfmp directory on the [Main2] main server station.


72 

Enter the following:

cd /opt/nsp/nfmp ↵


73 

Enter the following:

chown nsp:nsp *.tar.gz ↵


74 

Enter the following:

ls *.tar.gz ↵

The data backup files are listed.


75 

For each listed file, enter the following:

tar -xf filename.tar.gz -C /opt/nsp/nfmp/ ↵

where filename is a backup timestamp in the format MM-DD-hh-mm


76 

Enter the following:

rm -f *.tar.gz ↵


77 

Enter the following:

chown -R nsp:nsp /opt/nsp/nfmp/lte ↵


78 

Enter the following:

chown -R nsp:nsp /opt/nsp/nfmp/nebackup ↵


79 

Enter the following:

chown -R nsp:nsp /opt/nsp/nfmp/nelogs ↵


80 

Enter the following:

chown -R nsp:nsp /opt/nsp/nfmp/nesoftware ↵


81 

Enter the following:

chown -R nsp:nsp /opt/nsp/nfmp/os ↵


82 

Enter the following:

chown -R nsp:nsp /opt/nsp/nfmp/server/script/savedResults ↵


Enable Windows Active Directory access
 
83 

If you intend to use Windows Active Directory, or AD, for single-sign-on client access, you must configure LDAP remote authentication for AD; otherwise,go to Step 102.

Open the following file as a reference for use in subsequent steps:

/opt/nsp/os/install/examples/config.yml

Note: Consider the following.

  • The NFM-P does not assign a default user group to users of a remote authentication source that you define for Windows AD; the authentication source must provide the user group attributes.

  • Windows AD supports the following LDAP server types for remote authentication:

    AD—The user group of an AD user is derived from the group_base_dn attribute in the server configuration; group search filters are not supported.

    AUTHENTICATED—The server configuration must include bind credentials; group search filters are supported. After NFM-P initialization, you add the AD server bind credentials to the NSP password vault using the NSP Session Manager REST API.


84 

Locate the section that begins with the following lines:

#   ldap:

#     enabled: true

#     servers:

#       - type: AUTHENTICATED/AD/ANONYMOUS

#         url: ldaps://ldap.example.com:636

#         security: SSL/STARTTLS/NONE


85 

Open the following file using a plain-text editor such as vi:

/opt/nsp/os/install/config.json


86 

Locate the section that begins with the following line:

"sso": {

The section has one subsection for each type of SSO access.

Note: You can enable multiple remote authentication methods such as LDAP and RADIUS in the config.json file, or by using the NFM-P GUI. Using the GUI also allows you to specify the order in which the methods are tried during login attempts; however, no ordering is applied to multiple methods enabled in the config.json file.


87 

In the sso section, create an ldap subsection as shown below using the parameter names from the ldap section of config.yml and the required values for your configuration.

The following example shows the LDAP configuration for two AD servers:

    "ldap": {
      "enabled": true,
      "servers": [
        {
          "type": "auth_type",
          "url": "ldaps://server1:port",
          "server1_parameter_1": "value",
          "server1_parameter_2": "value",
          .
          .
          "server1_parameter_n": "value",
          },
        {
          "type": "auth_type",
          "url": "ldaps://server2:port",
          "server2_parameter_1": "value",
          "server2_parameter_2": "value",
          .
          .
          "server2_parameter_n": "value",
          },
      }]
    }

where auth_type is AD or AUTHENTICATED


88 

Save and close the files.


89 

Enter the following:

samconfig -m main ↵

The following is displayed:

Start processing command line inputs...

<main> 


90 

Enter the following:

<main> apply ↵

The AD LDAP configuration is applied.


91 

Enter the following:

<main> exit ↵

The samconfig utility closes.


Enable CAC access
 
92 

If you do not intend to enable Common Access Card, or CAC, technology for NFM-P client access, go to Step 102.


93 

Download the federationmetadata.xml from the following ADFS link:

https://ADFS_server_name/FederationMetadata/2007-06/federationmetadata.xml

where ADFS_server_name is the ADFS server FQDN


94 

Add an ADFS server entry to the /etc/hosts file on the main server.

  1. Open the /etc/hosts file using a plain-text editor such as vi.

  2. Add the following line below the line that contains the main server IP address:

    IP_address FQDN

    where

    IP_address is the IP address of the ADFS server

    FQDN is the FQDN of the ADFS server

  3. Save and close the file.


95 

In order to enable CAC for client access, you must configure Active Directory Federation Services, or ADFS.

Open the following file using a plain-text editor such as vi:

/opt/nsp/os/install/config.json


96 

In the sso section, create an saml2 subsection as shown below using the parameter names from the saml2 section of config.yml and the required values for your configuration.

The following example shows the ADFS configuration.

Note: You must preserve the lead spacing of each line.

  "sso" : {

    "saml2": {

       "enabled": true,

       "service_provider_entity_id": "NFM-P_identifier",

       "service_provider_metadata_filename": "casmetadata.xml",

       "maximum_authentication_lifetime": 3600,

       "accepted_skew": 300,

       "destination_binding": "urn:oasis:names:tc:SAML:2.0:bindings:HTTP-Redirect",

       "identity_provider_metadata_path": "ADFS_metadata_file",

       "authn_context_class_ref": "urn:oasis:names:tc:SAML:2.0:ac:classes:TLSClient",

       "authn_context_comparison_type": "minimum",

       "name_id_policy_format": "urn:oasis:names:tc:SAML:1.1:nameid-format:unspecified",

       "force_auth": true,

       "passive": false,

       "wants_assertions_signed": false,

       "wants_responses_signed": false,

       "all_signature_validation_disabled": false,

       "sign_service_provider_metadata": false,

       "principal_id_attribute": "UPN",

       "use_name_qualifier": false,

       "provider_name": "ADFS_server_URI",

       "requested_attributes": [{

         "name": "http://schemas.xmlsoap.org/ws/2005/05/identity/claims/emailaddress",

          "friendly_name": "E-Mail Address",

          "name_format": "urn:oasis:names:tc:SAML:2.0:attrname-format:uri",

          "required": false

      } ],

       "mapped_attributes": [{

           "name": "http://schemas.xmlsoap.org/claims/Group",

           "mapped_to": "authorizationProfile"

      }, {

           "name": "http://schemas.xmlsoap.org/ws/2005/05/identity/claims/upn",

           "mapped_to": "upn"

      } ]

    },


97 

Configure the following parameters; leave all other parameters at the default:

  • "service_provider_entity_id": "NFM-P_identifier"

  • "identity_provider_metadata_path": "ADFS_metadata_file"

  • "provider_name": "ADFS_server_name"

NFM-P_identifier is the unique ADFS Relying Trust Party identifier

ADFS_metadata_file is the absolute path of the ADFS metadata XML file, for example, /opt/federationmetadata.xml

ADFS_server_name is the ADFS server FQDN


98 

Save and close the files.


99 

Enter the following:

samconfig -m main ↵

The following is displayed:

Start processing command line inputs...

<main> 


100 

Enter the following:

<main> apply ↵

The ADFS configuration is applied.


101 

Enter the following:

<main> exit ↵

The samconfig utility closes.


Configure WS-NOC integration
 
102 

If the NFM-P is integrated with a WS-NOC system, open the following file with a plain-text editor such as vi:

/opt/nsp/os/install/examples/config.json

Otherwise, go to Step 112.


103 

Copy the following section:

  "nfmt": {

    "primary_ip": "",

    "standby_ip": "",

    "username": "",

    "password": "",

    "cert_provided": false

  },


104 

Close the file.


105 

Open the following file with a plain-text editor such as vi:

/opt/nsp/os/install/config.json


106 

Paste in the copied section.


107 

Configure the required parameters to enable the WS-NOC integration:

  • primary_ip—the primary WS-NOC server IP address

  • standby_ip—the standby WS-NOC server IP address

  • username—the username required for WS-NOC access

  • password—the password required for WS-NOC access

  • cert_provided—whether a TLS certificate is used


108 

Save and close the file.


109 

Enter the following:

samconfig -m main ↵

The following is displayed:

Start processing command line inputs...

<main> 


110 

Enter the following:

<main> apply ↵

The configuration is applied.


111 

Enter the following:

<main> exit ↵

The samconfig utility closes.


Stop NSP analytics servers, NSP Flow Collectors
 
112 

If the system includes one or more NSP analytics servers, stop each analytics server.

  1. Log in to the analytics server station as the nsp user.

  2. Open a console window.

  3. Enter the following:

    bash$ /opt/nsp/analytics/bin/AnalyticsAdmin.sh stop ↵

The following is displayed:

Stopping Analytics Application

When the analytics server is completely stopped, the following message is displayed:

Analytics Application is not running


113 

If the system includes one or more NSP Flow Collector Controllers and Flow Collectors, stop each NSP Flow Collector Controller.

Note: If the NSP Flow Collector Controller is collocated on a station with an NSP Flow Collector, stopping the NSP Flow Collector Controller also stops the Flow Collector.

  1. Log in to the NSP Flow Collector Controller station as the nsp user.

  2. Open a console window.

  3. Enter the following:

    bash$ /opt/nsp/flow/fcc/bin/flowCollectorController.bash stop ↵

    The NSP Flow Collector Controller stops.


114 

If the system includes one or more NSP Flow Collectors that are not collocated on a station with a Flow Collector Controller, stop each such NSP Flow Collector.

  1. Log in to the NSP Flow Collector station as the nsp user.

  2. Open a console window.

  3. Enter the following:

    bash$ /opt/nsp/flow/fc/bin/flowCollector.bash stop ↵

    The NSP Flow Collector stops.


Upgrade auxiliary servers [Aux2]
 
115 

If the system includes auxiliary servers, perform To upgrade a Release 22.6 or earlier NFM-P auxiliary server on each appropriate auxiliary server station [Aux2].


Verify auxiliary database synchronization
 
116 

If the system does not include redundant auxiliary database clusters, go to Step 121.


117 

If you are upgrading the first redundant auxiliary database cluster, you must verify the success of the most recent copy-cluster operation, which synchronizes the database data between the redundant clusters.

Note: You must not proceed to the next step until the copy-cluster operation is complete and successful.

Perform one of the following periodically to check the copy-cluster status.

  1. If the NFM-P is in a shared-mode NSP deployment, issue the following REST API call:

    Note: In order to issue a REST API call, you require a REST token; see this tutorial on the Network Developer Portal for information.

    GET https://address:8545/restconf/data/auxdb:/auxdb-agent

    where address is the advertised address of the primary NSP cluster

    The call returns a status of SUCCESS, as shown below, for a successfully completed copy-cluster operation:

    <HashMap>

          <auxdb-agent>

             <name>nspos-auxdb-agent</name>

             <application-mode>ACTIVE</application-mode>

             <copy-cluster>

                <source-cluster>cluster_M</source-cluster>

                <target-cluster>cluster_N</target-cluster>

                <time-started>timestamp</time-started>

                <status>SUCCESS</status>

             </copy-cluster>

          </auxdb-agent>

    </HashMap>

  2. If the NFM-P is not in a shared-mode NSP deployment and you are upgrading from NSP 21.9 or earlier, enter the following as the root user on the primary main server station [Main1]:

    grep "The copy cluster state has changed.*to.*SUCCESS" /opt/nsp/os/auxdb-agent/logs/nspos-auxdb-agent.log ↵

    The command returns an output line like the following for a successfully completed copy-cluster operation:

    <timestamp><I><server><ClusterManagerExecutorPool[0]><com.nokia.nsp.nspos.auxdb.agent.cluster.ClusterManager>The copy cluster state has changed from [RUNNING] to [SUCCESS] - [Completed..]

    If no output is displayed, a copy-cluster operation may be in progress.

  3. If the NFM-P is not in a shared-mode NSP deployment and you are upgrading from NSP 21.11 or later, enter the following as the root user on the primary main server station [Main1]:

    /opt/nsp/os/nspd/nspdctl auxdb agent-status ↵

    The command returns output like the following for a successfully completed copy-cluster operation:

         DC-ROLE HOST APPLICATION-MODE

         active leader 203.0.113.101 ACTIVE

         Copy Cluster Details

         SOURCE TARGET TIME-STARTED STATUS

         cluster_1 cluster_2 2022-03-14T15:09:26.535Z SUCCESS


Enable maintenance mode on auxiliary database agent
 
118 

Perform one of the following to enable nspos-auxdb-agent maintenance mode.

  1. If the NFM-P is in a shared-mode NSP deployment, perform the following steps.

    1. Log in as the root user on the NSP cluster host in the primary data center.

    2. Enter the following to set the nspos-auxdb-agent mode to maintenance:

      kubectl patch configmap/nspos-auxdb-agent-overrides -n namespace --type=merge -p '{"data":{"nspos-auxdb-agent-overrides.json":"{\"auxDbAgent\":{\"config\":{\"maintenance-mode\":true}}}"}}' ↵

      where namespace is the nspos-auxdb-agent namespace

    3. Enter the following to restart the nspos-auxdb-agent pod:

      kubectl delete -n namespace pod `kubectl describe -n namespace pods | grep -P ^^Name: | grep -oP nspos-auxdb-agent[-a-zA-Z0-9]+`↵

      where namespace is the nspos-auxdb-agent namespace

    4. Issue the following REST API call against the primary NSP cluster to verify that the agent is in maintenance mode:

      NOTE: In order to issue a REST API call, you require a REST token; see this tutorial on the Network Developer Portal for information.

      GET https://address:8545/restconf/data/auxdb:/auxdb-agent

      where address is the advertised address of the primary NSP cluster

      The call returns information like the following:

      {

          "auxdb-agent": {

              "name": "nspos-auxdb-agent",

              "application-mode": "MAINTENANCE",

              "copy-cluster": {

                  "source-cluster": "cluster_2",

                  "target-cluster": "cluster_1",

                  "time-started": "timestamp",

                  "status": "SUCCESS"

              }

          }

      }

      The agent is in maintenance mode if the application-mode is MAINTENANCE, as shown in the example.

    5. Log in as the root user on the NSP cluster host in the standby data center.

    6. Enter the following to set the nspos-auxdb-agent mode to maintenance:

      kubectl patch configmap/nspos-auxdb-agent-overrides -n namespace --type=merge -p '{"data":{"nspos-auxdb-agent-overrides.json":"{\"auxDbAgent\":{\"config\":{\"maintenance-mode\":true}}}"}}' ↵

      where namespace is the nspos-auxdb-agent namespace

  2. If the NFM-P is not in a shared-mode NSP deployment and you are upgrading from Release 21.11 or later, perform the following steps.

    1. Log in as the root user on the NSP cluster host in the primary data center.

    2. Enter the following to set the nspos-auxdb-agent mode to maintenance:

      sed -i -r 's/("maintenance-mode"\s*:\s*)false/\1true/g' /opt/nsp/os/auxdb-agent/conf/nspos-auxdb-agent-overrides.json ↵

    3. Enter the following to verify that the nspos-auxdb-agent is in maintenance mode:

      /opt/nsp/os/nspd/nspdctl auxdb agent-status ↵

      DC-ROLE         HOST             APPLICATION-MODE

      active leader   203.0.113.101    MAINTENANCE

      standby leader  203.0.113.102    inactive

      The agent is in maintenance mode if the APPLICATION-MODE of the active leader is MAINTENANCE, as shown in the example.

    4. Log in as the root user on the NSP cluster host in the standby data center.

    5. Enter the following to set the nspos-auxdb-agent mode to maintenance:

      sed -i -r 's/("maintenance-mode"\s*:\s*)false/\1true/g' /opt/nsp/os/auxdb-agent/conf/nspos-auxdb-agent-overrides.json ↵


Upgrade standby auxiliary database cluster
 
119 

If you are upgrading the first redundant auxiliary database cluster, perform the following steps to stop the database proxy on each station in each auxiliary database cluster.

  1. Enter the following sequence of commands as the root user on each auxiliary database station in the standby data center:

    systemctl stop nfmp-auxdbproxy.service ↵

    systemctl disable nfmp-auxdbproxy.service ↵

    The proxy stops, and is disabled.

  2. Enter the following sequence of commands as the root user on each auxiliary database station in the primary data center:

    systemctl stop nfmp-auxdbproxy.service ↵

    systemctl disable nfmp-auxdbproxy.service ↵

    The proxy stops, and is disabled.


120 

Perform To upgrade a Release 22.6 or earlier auxiliary database cluster to upgrade the standby auxiliary database cluster.


Stop and disable original primary main server [Main1]
 
121 

Stop the original primary main server.

Note: This step marks the beginning of the network management outage.

  1. Log in to the original primary main server station [Main1] as the nsp user.

  2. Open a console window.

  3. Enter the following:

    bash$ cd /opt/nsp/nfmp/server/nms/bin ↵

  4. Enter the following:

    bash$ ./nmsserver.bash stop ↵

  5. Enter the following:

    bash$ ./nmsserver.bash appserver_status ↵

    The server status is displayed; the server is fully stopped if the status is the following:

    Application Server is stopped

    If the server is not fully stopped, wait five minutes and then repeat this step. Do not perform the next step until the server is fully stopped.

  6. Enter the following to switch to the root user:

    bash$ su ↵

  7. If the NFM-P is not part of a shared-mode NSP deployment, enter the following to display the nspOS service status:

    nspdctl status ↵

    Information like the following is displayed.

    Mode:     DR

    Role:     redundancy_role

    DC-Role:  dc_role

    DC-Name:  dc_name

    Registry: IP_address:port

    State:    stopped

    Uptime:   0s

    SERVICE           STATUS

    service_a         inactive

    service_b         inactive

    service_c         inactive

    You must not proceed to the next step until all NSP services are stopped; if the State is not ‘stopped’, or the STATUS indicator of each listed service is not ‘inactive’, repeat this substep.


122 

Disable the automatic main server startup so that the main server does not start in the event of a power disruption during the upgrade.

  1. Enter the following:

    systemctl disable nspos-nspd.service ↵

  2. Enter the following:

    systemctl disable nfmp-main-config.service ↵

  3. Enter the following:

    systemctl disable nfmp-main.service ↵


Upgrade NSP Flow Collector Controllers, Flow Collectors
 
123 

If the system includes one or more NSP Flow Collectors, upgrade each NSP Flow Collector Controller and Flow Collector as described in NSP Flow Collector and Flow Collector Controller upgrade from Release 22.6 or earlier.


Stop auxiliary servers [Aux1]
 
124 

If the system includes auxiliary servers, perform the following steps on each [Aux1] auxiliary server station.

  1. Log in to the auxiliary server station as the nsp user.

  2. Open a console window.

  3. Enter the following:

    bash$ /opt/nsp/nfmp/auxserver/nms/bin/auxnmsserver.bash auxstop ↵

    The auxiliary server stops.


Stop original primary main database [DB1]
 
125 

Log in to the original primary main database [DB1] station as the root user.


126 

Open a console window.


127 

Stop and disable the Oracle proxy and main database services.

  1. Enter the following to stop the Oracle proxy:

    systemctl stop nfmp-oracle-proxy.service ↵

  2. Enter the following to disable the automatic Oracle proxy startup:

    systemctl disable nfmp-oracle-proxy.service ↵

  3. Enter the following to stop the main database:

    systemctl stop nfmp-main-db.service ↵

  4. Enter the following to disable the automatic database startup:

    systemctl disable nfmp-main-db.service ↵


128 

Perform the following steps.

  1. Open the /etc/fstab file using a plain-text editor such as vi.

  2. Locate the tmpfs file system entry.

  3. Remove the noexec option so that the entry reads as follows:

    tmpfs /dev/shm tmpfs nodev 0 0

  4. Save and close the /etc/fstab file.

  5. Enter the following to remount the /dev/shm partition:

    mount -o remount /dev/shm ↵


Upgrade auxiliary database, if not redundant
 
129 

If the system does not include an auxiliary database, go to Step 133.


130 

If the system includes a standalone auxiliary database, perform the following steps.

  1. Perform To upgrade a Release 22.6 or earlier auxiliary database cluster.

  2. Go to Step 133.


Enable maintenance mode for auxiliary database agent
 
131 

If the system includes redundant auxiliary database clusters, and the NFM-P is not in a shared-mode NSP deployment, enter the following as the root user on the newly upgraded main server [Main2]:

sed -i -r 's/("maintenance-mode"\s*:\s*)false/\1true/g' /opt/nsp/os/auxdb-agent/conf/nspos-auxdb-agent-overrides.json ↵

The auxiliary database cluster enters maintenance mode within approximately one minute.


Stop former primary auxiliary database cluster
 
132 

If the system includes redundant auxiliary database clusters, perform the following steps on one station in the upgraded former primary cluster.

  1. Log in as the root user.

  2. Open a console window.

  3. Enter the following:

    cd /opt/nsp/nfmp/auxdb/install/bin ↵

  4. Enter the following to stop the auxiliary database:

    ./auxdbAdmin.sh stop ↵

  5. Enter the following to display the auxiliary database status:

    ./auxdbAdmin.sh status ↵

    Information like the following is displayed:

    Database status

     Node       | Host          | State | Version | DB

    ------------+---------------+-------+---------+-------

     node_1 internal_IP_1 | STATE | version | db_name

     node_2 | internal_IP_2 | STATE | version | db_name

    .

    .

    .

     node_n | internal_IP_n | STATE | version | db_name

          Output captured in log_file

    The cluster is stopped when each STATE entry reads DOWN.

  6. Repeat substep 5 periodically until the cluster is stopped.

    Note: You must not proceed to the next step until the cluster is stopped.


Start new primary main server [Main2]
 
133 
CAUTION 

CAUTION

Service Disruption

The new primary database [DB2] must be upgraded and running before you start the new primary main server [Main2], or the main server initialization may fail.

If you perform the new primary main server and database upgrades concurrently, do not perform this step until the database upgrade is complete.

CAUTION 

CAUTION

Service Disruption

An NFM-P system upgrade is not complete until each main server performs crucial post-upgrade tasks during initialization.

Before you attempt an operation that requires a server shutdown, you must ensure that each main server is completely initialized, or the operation fails.

Start the new primary main server [Main2].

Note: You cannot start a main server unless the main server configuration includes a current and valid license. You can use samconfig to specify the license file, or import a license, as described in the NSP System Administrator Guide.

  1. Log in as the nsp user on the new primary main server station [Main2].

  2. Enter the following:

    bash$ cd /opt/nsp/nfmp/server/nms/bin ↵

  3. Enter the following:

    bash$ ./nmsserver.bash start ↵

  4. Enter the following:

    bash$ ./nmsserver.bash appserver_status ↵

    The server status is displayed; the server is fully initialized if the status is the following:

    Application Server process is running.  See nms_status for more detail.

    If the server is not fully initialized, wait five minutes and then repeat this step. Do not perform the next step until the server is fully initialized.

Note: This marks the end of the network management outage.


134 

If you have enabled CAC for NFM-P client access, download the casmetadata.xml file from the following URL, and then import the file into the ADFS server relying-trust-party:

https://server/cas/sp/metadata

where server is the main server IP address or hostname

After the download, the casmetadata.xml file is available in the following directory on the main server:

/opt/nsp/os/tomcat/conf/cas/saml


135 

If you have enabled Windows Active Directory access using the AUTHENTICATED type of LDAP server, perform the following steps.

  1. Use the NSP Session Manager REST API to add the LDAP server bind credentials; see the Network Developer Portal for information.

  2. If the NFM-P is not part of a shared-mode NSP deployment, enter the following to restart the local nspos-tomcat service:

    Note: The service restart may take a few minutes, during which NFM-P GUI and REST client access is degraded. General NFM-P operation is unaffected.

    systemctl restart nspos-tomcat ↵


136 

Specify the memory requirement for GUI clients based on the type of network that the NFM-P is to manage.

  1. Enter the following:

    bash$ ./nmsdeploytool.bash clientmem -option

    where option is one of the following:

    • m—medium, for management of limited-scale network

    • l—large, for a network of 15 000 or more NEs

  2. Record the setting, which is not preserved through an upgrade, for future use.

  3. Enter the following to commit the configuration change:

    bash$ ./nmsdeploytool.bash deploy ↵


Start auxiliary servers [Aux2]
 
137 

If the NFM-P system includes auxiliary servers, start each appropriate auxiliary server [Aux2].

  1. Log in to the auxiliary server station as the nsp user.

  2. Open a console window.

  3. Enter the following:

    bash$ cd /opt/nsp/nfmp/auxserver/nms/bin ↵

  4. Enter the following:

    bash$ ./auxnmsserver.bash auxstart ↵

    The auxiliary server starts.


Activate upgraded former standby auxiliary database cluster
 
138 

If the system does not include redundant auxiliary database clusters, go to Step 141.


139 

Perform the following steps on each station in the upgraded former standby auxiliary database cluster.

  1. Log in as the root user on the station.

  2. Open a console window.

  3. Enter the following sequence of commands to enable the database services:

    systemctl enable nspos-auxdb.service ↵

    systemctl enable nspos-auxdbproxy.service ↵

    systemctl enable vertica_agent.service ↵

    systemctl enable verticad.service ↵

    The services are enabled.

  4. Enter the following to start the database proxy:

    systemctl start nspos-auxdbproxy.service ↵

    The proxy starts.


140 

Perform one of the following to activate the former standby auxiliary database cluster, after which the cluster assumes the primary role.

  1. If the NFM-P is in a shared-mode NSP deployment, issue the following REST API call:

    Note: In order to issue a REST API call, you require a REST token; see this tutorial on the Network Developer Portal for information.

    POST https://{{address}}:8545/restconf/data/auxdb:/clusters/cluster=cluster_N/activate

    where

    address is the advertised address of the primary NSP cluster

    N is the auxiliary database cluster number

    The following is the request body:

    {

      "auxdb:input" : {

        "force": true

      }

    }

    The cluster is activated.

  2. If the NFM-P is not in a shared-mode NSP deployment, enter the following as the root user on the new primary main server station:

    nspdctl auxdb activate cluster_N --force ↵

    where N is the auxiliary database

    A message like the following is displayed:

    Auxiliary database activation request submitted for [cluster_N]


Upgrade analytics servers
 
141 

If the system includes one or more NSP analytics servers, upgrade each analytics server as described in NSP analytics server upgrade from Release 22.6 or earlier.


Enable GUI client
 
142 

You require an NFM-P GUI client to complete the procedure; see the following for information:

Note: A client delegate server installation typically takes more time than the other options. A single-user client or client delegate server upgrade is recommended if your maintenance period is limited.


Test upgraded system using GUI client
 
143 

When the new primary main server [Main2] is started, use a newly installed or upgraded GUI client to perform sanity testing of the new primary main server and database.

Note: To back out of the upgrade and return the original primary main server [Main1] and database [DB1] to service, you can do so by stopping the new primary main server [Main2] and database [DB2] and restarting the original primary main server [Main1] and database [DB1].


Uninstall original primary database [DB1]
 
144 

Enter the following to uninstall the original primary main database:

yum remove nsp-nfmp-main-db nsp-nfmp-oracle ↵

The yum utility resolves any dependencies and displays the following prompt:

Installed size: nn G 

Is this ok [y/N]: 


145 

Enter y. The following is displayed as the packages are removed:

Downloading Packages:

Running transaction check

Transaction check succeeded.

Running transaction test

Transaction test succeeded.

Running transaction check

Uninstalling the NFM-P package...

As each package removal completes, the following is displayed:

Complete!


Install new standby main database [DB1]
 
146 

If you are re-using the primary main database [DB1] station, recommission the station according to the platform specifications in this guide and in the NSP Planning Guide.

For information about deploying the RHEL OS using an NSP OEM disk image, see NSP disk-image deployment.

Note: After the upgrade, the station is the new standby main database station.


147 

Log in as the root user on the station that is commissioned as the main database [DB1] station.


148 

Perform one of the following.

  1. If the main server and database are collocated on one station, perform the following steps.

    1. Enter the following:

      mkdir /opt/importConfigs ↵

    2. Transfer the mainserverBackupConfigs.tar.gz file created in Step 32 of To prepare for an NFM-P system upgrade from Release 22.6 or earlier to the /opt/importConfigs directory.

    3. Transfer the following downloaded installation files to an empty directory on the collocated station:

      • nsp-nfmp-oracle-R.r.p-rel.v.rpm

      • nsp-nfmp-main-db-R.r.p-rel.v.rpm

      • nsp-nfmp-nspos-R.r.p.rpm

      • nsp-nfmp-jre-R.r.p-rel.v.rpm

      • nsp-nfmp-config-R.r.p-rel.v.rpm

      • nsp-nfmp-main-server-R.r.p.rpm

      Note: In subsequent steps, the directory is called the NFM-P software directory.

  2. If the main server and database are on separate stations, transfer the following downloaded installation files to an empty directory on the main database station:

    • nsp-nfmp-jre-R.r.p-rel.v.rpm

    • nsp-nfmp-config-R.r.p-rel.v.rpm

    • nsp-nfmp-oracle-R.r.p-rel.v.rpm

    • nsp-nfmp-main-db-R.r.p-rel.v.rpm

    • nsp-nfmp-nodeexporter-R.r.p-rel.v.rpm, if downloaded

    Note: In subsequent steps, the directory is called the NFM-P software directory.


149 

Transfer the following downloaded file to an empty directory on the main database station:

  • OracleSw_PreInstall.sh


150 

Open a console window.


151 

Navigate to the directory that contains the OracleSw_PreInstall.sh file.


152 

Enter the following:

chmod +x OracleSw_PreInstall.sh ↵


153 

Enter the following:

./OracleSw_PreInstall.sh ↵

Note: A default value is displayed in brackets []. To accept the default, press ↵.

Note: If you specify a value other than the default, you must record the value for use when the OracleSw_PreInstall.sh script is run during a software upgrade, or when the Oracle management user information is required by technical support.

The following prompt is displayed:

This script will prepare the system for a new install/restore of an NFM-P Version R.r Rn database.

Do you want to continue? [Yes/No]:


154 

Enter Yes. The following prompt is displayed:

Enter the Oracle dba group name [group]:


155 

Enter a group name.

Note: To reduce the complexity of subsequent software upgrades and technical support activities, it is recommended that you accept the default.

The following messages and prompt are displayed:

Creating group group if it does not exist...

done

Enter the Oracle user name:


156 

Enter a username.

Note: To reduce the complexity of subsequent software upgrades and technical support activities, it is recommended that you accept the default.

The following messages and prompt are displayed:

Oracle user [username] new home directory will be [/opt/nsp/nfmp/oracle19].

Checking or Creating the Oracle user home directory /opt/nsp/nfmp/oracle19...

Checking user username...

Adding username...

Changing ownership of the directory /opt/nsp/nfmp/oracle19 to username:group.

About to unlock the UNIX user [username]

Unlocking password for user username.

passwd: Success

Unlocking the UNIX user [username] completed

Please assign a password to the UNIX user username ..

New Password:


157 

Enter a password. The following prompt is displayed:

Re-enter new Password:


158 

Re-enter the password. The following is displayed if the password change is successful:

passwd: password successfully changed for username

The following message and prompt are displayed:

Specify whether an NFM-P Main Server will be installed on this workstation.

The database memory requirements will be adjusted to account for the additional load.

Will the database co-exist with an NFM-P Main Server on this workstation [Yes/No]:


159 

Enter Yes or No, as required.

Messages like the following are displayed as the script execution completes:

INFO: About to set kernel parameters in /etc/sysctl.conf...

INFO: Completed setting kernel parameters in /etc/sysctl.conf...

INFO: About to change the current values of the kernel parameters

INFO: Completed changing the current values of the kernel parameters

INFO: About to set ulimit parameters in /etc/security/limits.conf...

INFO: Completed setting ulimit parameters in /etc/security/limits.conf...

INFO: Completed running Oracle Pre-Install Tasks, you *MUST* reboot your box.


160 

When the script execution is complete, enter the following to reboot the station:

systemctl reboot ↵

The station reboots.


161 

When the reboot is complete, log in as the root user on the station that is commissioned as the main database [DB1] station.


162 

Open a console window.


163 

Navigate to the NFM-P software directory.

Note: Ensure that the directory contains only the installation files.


164 

Enter the following:

chmod +x * ↵


165 

Enter the following:

dnf install *.rpm ↵

The dnf utility resolves any package dependencies, and displays the following prompt:

Total size: nn G

Installed size: nn G 

Is this ok [y/d/N]: 


166 

Enter y. The following and the installation status are displayed as each package is installed:

Downloading Packages:

Running transaction check

Transaction check succeeded.

Running transaction test

Transaction test succeeded.

Running transaction check

The package installation is complete when the following is displayed:

Complete!


167 

Configure the database as a standby database; see NFM-P samconfig utility for information about using samconfig.

  1. Enter the following:

    samconfig -m db ↵

    The following is displayed:

    Start processing command line inputs...

    <db> 

  2. Enter the following:

    <db> configure type standby ↵

    The prompt changes to <db configure>.

  3. Enter the following:

    <db configure> ip address

    where address is the IP address of this database

  4. Enter the following:

    <db configure> redundant ip address

    where address is the IP address of the new primary database [DB2]

    The prompt changes to <db configure redundant>.

  5. Enter the following:

    <db configure redundant> instance instance_name

    where instance_name is the instance name of the new primary database [DB2]

  6. Enter the following:

    <db configure redundant> back ↵

    The prompt changes to <db configure>.

  7. Enter the following:

    <db configure> passwords sys password

    where password is the database SYS user password]

    The prompt changes to <db configure passwords>.

  8. Enter the following:

    <db configure passwords> back ↵

    The prompt changes to <db configure>.


168 

Verify the database configuration.

  1. Enter the following:

    <db configure> show-detail ↵

    The database configuration is displayed.

  2. Review each parameter to ensure that the value is correct; see NFM-P samconfig utility for information about using the samconfig utility.

  3. Configure one or more parameters, if required.

  4. When you are certain that the configuration is correct, enter the following:

    <db configure> back ↵

    The prompt changes to <db>.


169 

Enter the following to apply the configuration and begin the database creation:

<db> apply ↵

The database creation begins, and progress messages are displayed.

The following is displayed when the database creation is complete:

DONE

db configurations updated.


170 

When the database creation is complete, enter the following:

<db> exit ↵

The samconfig utility closes.


Reinstantiate standby database
 
171 

Log in to an NFM-P GUI client as the admin user.


172 

Choose Administration→System Information from the main menu. The System Information form opens.


173 

Click Re-Instantiate Standby.


174 

Click Yes to confirm the action. The reinstantiation begins, and the GUI status bar displays reinstantiation information.

Note: Database reinstantiation takes considerable time if the database contains a large amount of statistics data.

You can also use the System Information form to monitor the reinstantiation progress. The Last Attempted Standby Re-instantiation Time is the start time; the Standby Re-instantiation State changes from In Progress to Success when the reinstantiation is complete.


175 

When the reinstantiation is complete, close the System Information form.


Upgrade former primary auxiliary database cluster
 
176 

If the system includes redundant auxiliary database clusters, perform To upgrade a Release 22.6 or earlier auxiliary database cluster on the former primary auxiliary database cluster.


Upgrade original primary main server [Main1]
 
177 

If the main server [Main1] and database [DB1] are on separate stations, and you are re-using the main server station, recommission the station according to the platform specifications in this guide and in the NSP Planning Guide.

For information about deploying the RHEL OS using an NSP OEM disk image, see NSP disk-image deployment.

Note: After the upgrade, the station is the new standby main server station.


178 

Log in as the root user on the station that is commissioned as the main server [Main1] station.


179 

Open a console window.


180 

If the main server and database are on separate stations, perform the following steps.

  1. Enter the following:

    mkdir /opt/importConfigs ↵

  2. Transfer the mainserverBackupConfigs.tar.gz file created in Step 32 of To prepare for an NFM-P system upgrade from Release 22.6 or earlier to the /opt/importConfigs directory.


181 

Perform one of the following.

  1. If the main server and database are collocated on one station, go to Step 186.

  2. If the main server and database are on separate stations, transfer the following downloaded installation files to an empty directory on the main server station:

    • nsp-nfmp-nspos-R.r.p.rpm

    • nsp-nfmp-jre-R.r.p-rel.v.rpm

    • nsp-nfmp-config-R.r.p-rel.v.rpm

    • nsp-nfmp-main-server-R.r.p.rpm

    • nsp-nfmp-nodeexporter-R.r.p-rel.v.rpm, if downloaded

    Note: In subsequent steps, the directory is called the NFM-P software directory.


182 

Navigate to the NFM-P software directory.


183 

Enter the following:

chmod +x * ↵


184 

Enter the following:

dnf install *.rpm ↵

The dnf utility resolves any package dependencies, and displays the following prompt:

Total size: nn G

Installed size: nn G 

Is this ok [y/d/N]: 


185 

Enter y. The following and the installation status are displayed as each package is installed:

Downloading Packages:

Running transaction check

Transaction check succeeded.

Running transaction test

Transaction test succeeded.

Running transaction check

The package installation is complete when the following is displayed:

Complete!


Configure new standby main server [Main1]
 
186 

Enter the following; see NFM-P samconfig utility for information about using samconfig:

Note: Regardless of whether you intend to modify the main server configuration, you must apply the main server configuration, as described in the following steps.

samconfig -m main ↵

The following is displayed:

Start processing command line inputs...

<main>


187 

Enter the following:

<main> configure ↵

The prompt changes to <main configure>.


188 

To apply a new or updated NFM-P license, enter the following:

Note: You cannot start a main server unless the main server configuration includes a current and valid license. You can use samconfig to specify the license file in this step, or later import the license, as described in the NSP System Administrator Guide.

<main configure> license license_file

where license_file is the path and file name of the NSP license bundle


189 

Enter the following:

<main configure> database instance standby_instance back ↵

where standby_instance is the [DB2] database instance name, which is the standby Instance Name recorded in Step 44 of To prepare for an NFM-P system upgrade from Release 22.6 or earlier


190 

Enter the following:

<main configure> redundancy database instance primary_instance back ↵

where primary_instance is the [DB1] database instance name, which is the primary Instance Name recorded in Step 43 of To prepare for an NFM-P system upgrade from Release 22.6 or earlier

The prompt changes to <main configure redundancy>.


191 

Enter the following:

<main configure redundancy> back ↵

The prompt changes to <main configure>.


192 

Verify the main server configuration.

  1. Enter the following:

    <main configure> show ↵

    The main server configuration is displayed.

  2. Review each parameter to ensure that the value is correct; see NFM-P samconfig utility for information about using the samconfig utility.

  3. Configure one or more parameters, if required.

    Note: The NFM-P uses the database backup settings to initialize the database during installation only. To change the backup settings after installation, you must use the Database Manager form in the NFM-P client GUI, as described in the NSP System Administrator Guide.

  4. When you are certain that the configuration is correct, enter the following:

    <main configure> back ↵

    The prompt changes to <main>.


193 

Enter the following:

<main> apply ↵

The configuration is applied.


194 

Enter the following:

<main> exit ↵

The samconfig utility closes.

Note: This station is the new standby main server station.


195 

If the NFM-P is part of a shared-mode NSP system and you want to enable mTLS for internal Kafka authentication using two-way TLS, perform the following steps.

Note: Enabling mTLS for internal Kafka authentication is supported only in an NSP deployment that uses separate interfaces for internal and client communication.

Note: The parameter you must configure is displayed only if the ip-list parameter is set to a remote address.

Note: The parameter is configurable only if the secure and internal-certs parameters in the nspos section are set to true.

  1. Enter the following:

    samconfig -m main ↵

    The following is displayed:

    Start processing command line inputs...

    <main> 

  2. Enter the following:

    configure nspos mtls-kafka-enabled back ↵

  3. Enter the following:

    <main> apply ↵

    The configuration is applied.

  4. Enter the following:

    <main> exit ↵

    The samconfig utility closes.


Restore new standby main server [Main1] data files
 
196 

Transfer the main server data backup .tar.gz file set created in Step 34 of To prepare for an NFM-P system upgrade from Release 22.6 or earlier to the /opt/nsp/nfmp directory on the [Main1] main server station.


197 

Enter the following:

cd /opt/nsp/nfmp ↵


198 

Enter the following:

chown nsp:nsp *.tar.gz ↵


199 

Enter the following:

ls *.tar.gz ↵

The data backup files are listed.


200 

For each listed file, enter the following:

tar -xf filename.tar.gz -C /opt/nsp/nfmp/ ↵

where filename is a backup timestamp in the format MM-DD-hh-mm


201 

Enter the following:

rm -f *.tar.gz ↵


202 

Enter the following:

chown -R nsp:nsp /opt/nsp/nfmp/lte ↵


203 

Enter the following:

chown -R nsp:nsp /opt/nsp/nfmp/nebackup ↵


204 

Enter the following:

chown -R nsp:nsp /opt/nsp/nfmp/nelogs ↵


205 

Enter the following:

chown -R nsp:nsp /opt/nsp/nfmp/nesoftware ↵


206 

Enter the following:

chown -R nsp:nsp /opt/nsp/nfmp/os ↵


207 

Enter the following:

chown -R nsp:nsp /opt/nsp/nfmp/server/script/savedResults ↵


Enable Windows Active Directory access
 
208 

If you intend to use Windows Active Directory, or AD, for single-sign-on client access, you must configure LDAP remote authentication for AD; otherwise, go to Step 227.

Open the following file as a reference for use in subsequent steps:

/opt/nsp/os/install/examples/config.yml

Note: Consider the following.

  • The NFM-P does not assign a default user group to users of a remote authentication source that you define for Windows AD; the authentication source must provide the user group attributes.

  • Windows AD supports the following LDAP server types for remote authentication:

    AD—The user group of an AD user is derived from the group_base_dn attribute in the server configuration; group search filters are not supported.

    AUTHENTICATED—The server configuration must include bind credentials; group search filters are supported. After NFM-P initialization, you add the AD server bind credentials to the NSP password vault using the NSP Session Manager REST API.


209 

Locate the section that begins with the following lines:

#   ldap:

#     enabled: true

#     servers:

#       - type: AUTHENTICATED/AD/ANONYMOUS

#         url: ldaps://ldap.example.com:636

#         security: SSL/STARTTLS/NONE


210 

Open the following file using a plain-text editor such as vi:

/opt/nsp/os/install/config.json


211 

Locate the section that begins with the following line:

"sso": {

The section has one subsection for each type of SSO access.

Note: You can enable multiple remote authentication methods such as LDAP and RADIUS in the config.json file, or by using the NFM-P GUI. Using the GUI also allows you to specify the order in which the methods are tried during login attempts; however, no ordering is applied to multiple methods enabled in the config.json file.


212 

In the sso section, create an ldap subsection as shown below using the parameter names from the ldap section of config.yml and the required values for your configuration.

The following example shows the LDAP configuration for two AD servers:

    "ldap": {
      "enabled": true,
      "servers": [
        {
          "type": "auth_type",
          "url": "ldaps://server1:port",
          "server1_parameter_1": "value",
          "server1_parameter_2": "value",
          .
          .
          "server1_parameter_n": "value",
          },
        {
          "type": "auth_type",
          "url": "ldaps://server2:port",
          "server2_parameter_1": "value",
          "server2_parameter_2": "value",
          .
          .
          "server2_parameter_n": "value",
          },
      }]
    }

where auth_type is AD or AUTHENTICATED


213 

Save and close the files.


214 

Enter the following:

samconfig -m main ↵

The following is displayed:

Start processing command line inputs...

<main> 


215 

Enter the following:

<main> apply ↵

The AD LDAP configuration is applied.


216 

Enter the following:

<main> exit ↵

The samconfig utility closes.


Enable CAC access
 
217 

If you do not intend to enable Common Access Card, or CAC, technology for NFM-P client access, go to Step 227.


218 

Download the federationmetadata.xml from the following ADFS link:

https://ADFS_server_name/FederationMetadata/2007-06/federationmetadata.xml

where ADFS_server_name is the ADFS server FQDN


219 

Add an ADFS server entry to the /etc/hosts file on the main server.

  1. Open the /etc/hosts file using a plain-text editor such as vi.

  2. Add the following line below the line that contains the main server IP address:

    IP_address FQDN

    where

    IP_address is the IP address of the ADFS server

    FQDN is the FQDN of the ADFS server

  3. Save and close the file.


220 

In order to enable CAC for client access, you must configure Active Directory Federation Services, or ADFS.

Open the following file using a plain-text editor such as vi:

/opt/nsp/os/install/config.json


221 

In the sso section, create an saml2 subsection as shown below using the parameter names from the saml2 section of config.yml and the required values for your configuration.

The following example shows the ADFS configuration.

Note: You must preserve the lead spacing of each line.

  "sso" : {

    "saml2": {

       "enabled": true,

       "service_provider_entity_id": "NFM-P_identifier",

       "service_provider_metadata_filename": "casmetadata.xml",

       "maximum_authentication_lifetime": 3600,

       "accepted_skew": 300,

       "destination_binding": "urn:oasis:names:tc:SAML:2.0:bindings:HTTP-Redirect",

       "identity_provider_metadata_path": "ADFS_metadata_file",

       "authn_context_class_ref": "urn:oasis:names:tc:SAML:2.0:ac:classes:TLSClient",

       "authn_context_comparison_type": "minimum",

       "name_id_policy_format": "urn:oasis:names:tc:SAML:1.1:nameid-format:unspecified",

       "force_auth": true,

       "passive": false,

       "wants_assertions_signed": false,

       "wants_responses_signed": false,

       "all_signature_validation_disabled": false,

       "sign_service_provider_metadata": false,

       "principal_id_attribute": "UPN",

       "use_name_qualifier": false,

       "provider_name": "ADFS_server_URI",

       "requested_attributes": [{

         "name": "http://schemas.xmlsoap.org/ws/2005/05/identity/claims/emailaddress",

          "friendly_name": "E-Mail Address",

          "name_format": "urn:oasis:names:tc:SAML:2.0:attrname-format:uri",

          "required": false

      } ],

       "mapped_attributes": [{

           "name": "http://schemas.xmlsoap.org/claims/Group",

           "mapped_to": "authorizationProfile"

      }, {

           "name": "http://schemas.xmlsoap.org/ws/2005/05/identity/claims/upn",

           "mapped_to": "upn"

      } ]

    },


222 

Configure the following parameters; leave all other parameters at the default:

  • "service_provider_entity_id": "NFM-P_identifier"

  • "identity_provider_metadata_path": "ADFS_metadata_file"

  • "provider_name": "ADFS_server_name"

NFM-P_identifier is the unique ADFS Relying Trust Party identifier

ADFS_metadata_file is the absolute path of the ADFS metadata XML file, for example, /opt/federationmetadata.xml

ADFS_server_name is the ADFS server FQDN


223 

Save and close the files.


224 

Enter the following:

samconfig -m main ↵

The following is displayed:

Start processing command line inputs...

<main> 


225 

Enter the following:

<main> apply ↵

The ADFS configuration is applied.


226 

Enter the following:

<main> exit ↵

The samconfig utility closes.


Configure WS-NOC integration
 
227 

If the NFM-P is integrated with an WS-NOC system, open the following file with a plain-text editor such as vi; otherwise, go to Step 237:

/opt/nsp/os/install/examples/config.json


228 

Copy the following section:

  "nfmt": {

    "primary_ip": "",

    "standby_ip": "",

    "username": "",

    "password": "",

    "cert_provided": false

  },


229 

Close the file.


230 

Open the following file with a plain-text editor such as vi:

/opt/nsp/os/install/config.json


231 

Paste in the copied section.


232 

Configure the required parameters to enable the WS-NOC integration:

  • primary_ip—the primary WS-NOC server IP address

  • standby_ip—the standby WS-NOC server IP address

  • username—the username required for WS-NOC access

  • password—the password required for WS-NOC access

  • cert_provided—whether a TLS certificate is used


233 

Save and close the file.


234 

Enter the following:

samconfig -m main ↵

The following is displayed:

Start processing command line inputs...

<main> 


235 

Enter the following:

<main> apply ↵

The configuration is applied.


236 

Enter the following:

<main> exit ↵

The samconfig utility closes.


Start new standby main server [Main1]
 
237 

Start the new standby main server [Main1].

Note: You cannot start a main server unless the main server configuration includes a current and valid license. You can use samconfig to specify the license file, or import a license, as described in the NSP System Administrator Guide.

  1. Enter the following to switch to the nsp user:

    su - nsp ↵

  2. Open a console window.

  3. Enter the following:

    bash$ cd /opt/nsp/nfmp/server/nms/bin ↵

  4. Enter the following:

    bash$ ./nmsserver.bash start ↵

  5. Enter the following:

    bash$ ./nmsserver.bash appserver_status ↵

    The server status is displayed; the server is fully initialized if the status is the following:

    Application Server process is running.  See nms_status for more detail.

    If the server is not fully initialized, wait five minutes and then repeat this step. Do not perform the next step until the server is fully initialized.


238 

If you have enabled CAC for NFM-P client access, download the casmetadata.xml file from the following URL, and then import the file into the ADFS server relying-trust-party:

https://server/cas/sp/metadata

where server is the main server IP address or hostname

After the download, the casmetadata.xml file is available in the following directory on the main server:

/opt/nsp/os/tomcat/conf/cas/saml


239 

If you have enabled Windows Active Directory access using the AUTHENTICATED type of LDAP server, perform the following steps.

  1. Use the NSP Session Manager REST API to add the LDAP server bind credentials; see the Network Developer Portal for information.

  2. If the NFM-P is not part of a shared-mode NSP deployment, enter the following to restart the local nspos-tomcat service:

    Note: The service restart may take a few minutes, during which NFM-P GUI and REST client access is degraded. General NFM-P operation is unaffected.

    systemctl restart nspos-tomcat ↵


240 

Specify the memory requirement for GUI clients based on the type of network that the NFM-P is to manage.

  1. Enter the following:

    bash$ ./nmsdeploytool.bash clientmem -option

    where option is one of the following:

    • m—medium, for management of limited-scale network

    • l—large, for a network of 15 000 or more NEs

  2. Record the setting, which is not preserved through an upgrade, for future use.

  3. Enter the following to commit the configuration change:

    bash$ ./nmsdeploytool.bash deploy ↵


241 

Close the console window.


Upgrade auxiliary servers [Aux1]
 
242 

If the system includes auxiliary servers, perform To upgrade a Release 22.6 or earlier NFM-P auxiliary server on each [Aux1] auxiliary server station.


Start auxiliary servers [Aux1]
 
243 

If the system includes auxiliary servers, perform the following steps on each [Aux1] auxiliary server station.

  1. Log in to the auxiliary server station as the nsp user.

  2. Open a console window.

  3. Enter the following:

    bash$ /opt/nsp/nfmp/auxserver/nms/bin/auxnmsserver.bash auxstart ↵

    The auxiliary server starts.


Disable maintenance mode for auxiliary database agents
 
244 

If the system does not include an auxiliary database, go to Step 248.


245 

If the system includes redundant auxiliary database clusters, perform one of the following to put each agent in active mode.

  1. If the NFM-P is in a shared-mode NSP deployment, perform the following steps.

    1. Log in as the root user on the NSP cluster host in the primary data center.

    2. Enter the following to set the nspos-auxdb-agent mode to active:

      kubectl patch configmap/nspos-auxdb-agent-overrides -n namespace --type=merge -p '{"data":{"nspos-auxdb-agent-overrides.json":"{\"auxDbAgent\":{\"config\":{\"maintenance-mode\":true}}}"}}' ↵

      where namespace is the nspos-auxdb-agent namespace

    3. Enter the following to restart the nspos-auxdb-agent:

      kubectl delete -n namespace pod `kubectl describe -n namespace pods | grep -P ^^Name: | grep -oP nspos-auxdb-agent[-a-zA-Z0-9]+`↵

      where namespace is the nspos-auxdb-agent namespace

    4. Log in as the root user on the NSP cluster host in the standby data center.

    5. Enter the following to set the nspos-auxdb-agent mode to active:

      kubectl patch configmap/nspos-auxdb-agent-overrides -n namespace --type=merge -p '{"data":{"nspos-auxdb-agent-overrides.json":"{\"auxDbAgent\":{\"config\":{\"maintenance-mode\":true}}}"}}' ↵

      where namespace is the nspos-auxdb-agent namespace

  2. If the NFM-P is not in a shared-mode NSP deployment, enter the following as the root user on the new primary main server [Main2]:

    sed -i -r 's/("maintenance-mode"\s*:\s*)true/\1false/g' /opt/nsp/os/auxdb-agent/conf/nspos-auxdb-agent-overrides.json ↵

    The cluster enters active mode within approximately one minute.


Verify auxiliary database status
 
246 

You must verify that the standalone or new primary auxiliary database cluster is in active mode.

  1. If the NFM-P is in a shared-mode NSP deployment, issue the following REST API call:

    Note: In order to issue a REST API call, you require a REST token; see this tutorial on the Network Developer Portal for information.

    GET /data/auxdb:/auxdb-agent HTTP/1.1

    Request body:

        Host: address:8545

        Content-Type: application/json

        Authorization: bearer_and_token_from_session_manager

    where address is the advertised address of the primary NSP cluster

    The cluster is in active mode if the REST response includes ACTIVE.

  2. If the NFM-P is not in a shared-mode NSP deployment, enter the following as the root user on the new primary main server [Main2]:

    /opt/nsp/os/nspd/nspdctl auxdb agent-status ↵

    A status message is displayed.

The cluster is in active mode if the message includes ACTIVE.


247 

Perform one of the following to verify the auxiliary database operation.

  1. If the NFM-P is in a shared-mode NSP deployment, issue the following REST API call:

    Note: In order to issue a REST API call, you require a REST token; see this tutorial on the Network Developer Portal for information.

    GET https://{{address}}:8545/restconf/data/auxdb:/clusters

    where address is the advertised address of the primary NSP cluster

    The call returns auxiliary database cluster status information like the following, which is the output for redundant clusters; if each mode and status value are not as shown below, contact technical support.

    <HashMap>

        <clusters>

            <cluster>

                <name>cluster_M</name>

                <mode>ACTIVE</mode>

                <status>UP</status>

                <nodes>

                    <external-ip>203.0.113.101</external-ip>

                    <internal-ip>10.1.2.101</internal-ip>

                    <status>UP</status>

                </nodes>

                <nodes>

                    <external-ip>203.0.113.102</external-ip>

                    <internal-ip>10.1.2.102</internal-ip>

                    <status>UP</status>

                </nodes>

                <nodes>

                    <external-ip>203.0.113.103</external-ip>

                    <internal-ip>10.1.2.103</internal-ip>

                    <status>UP</status>

                </nodes>

            </cluster>

            <cluster>

                <name>cluster_N</name>

                <mode>STANDBY</mode>

                <status>ON_STANDBY</status>

                <nodes>

                    <external-ip>203.0.113.104</external-ip>

                    <internal-ip>10.1.2.104</internal-ip>

                    <status>READY</status>

                </nodes>

                <nodes>

                    <external-ip>203.0.113.105</external-ip>

                    <internal-ip>10.1.2.105</internal-ip>

                    <status>READY</status>

                </nodes>

                <nodes>

                    <external-ip>203.0.113.106</external-ip>

                    <internal-ip>10.1.2.106</internal-ip>

                    <status>READY</status>

                </nodes>

            </cluster>

        </clusters>

    </HashMap>

  2. If the NFM-P is not in a shared-mode NSP deployment, enter the following as the root user on the primary main server [Main2]:

    nspdctl auxdb status ↵

    Cluster status information such as the following is displayed.

    Note: The Output for a standalone auxiliary database shows only one cluster.

    CLUSTER    DC-ROLE   STATE

    cluster_M  ACTIVE    UP

    NODE           INTERNAL IP   STATE

    203.0.113.101  10.1.2.101    UP

    203.0.113.102  10.1.2.102    UP

    203.0.113.103  10.1.2.103    UP

    CLUSTER    DC-ROLE   STATE

    cluster_N  STANDBY   ON_STANDBY

    NODE           INTERNAL IP    STATE

    203.0.113.104  10.1.2.104     READY

    203.0.113.105  10.1.2.105     READY

    203.0.113.106  10.1.2.106     READY

    If each STATE value is not as shown above, contact technical support.


Check post-upgrade disk space
 
248 

If you are performing a trial upgrade on a lab system in advance of a live upgrade, you must check the available capacity of the disk partitions on each component against the values recorded in Step 1.

Perform the following steps on each of the following stations:

  • main server

  • auxiliary server

  • main database

  • auxiliary database

  1. Log in to the station as the root user.

  2. Open a console window.

  3. Enter the following:

    df -kh ↵

    The usage information for each partition is displayed.

  4. Record the information for each NFM-P partition; see the tables in Chapter 2, NSP disk setup and partitioning for the partition names and required capacities.

  5. Compare the partition values with the values recorded in Step 1.

  6. If the disk usage on an NFM-P partition approaches 80% or has increased substantially, you may need to add disk capacity before you attempt the upgrade on a live system. Contact technical support for assistance.


Install or upgrade single-user GUI clients
 
249 

As required, install or upgrade additional single-user GUI clients; see the following for information:


Install or upgrade client delegate servers
 
250 

As required, install or upgrade client delegate servers; see the following for information:


Stop PKI server
 
251 

If no other components are to be deployed, stop the PKI server by entering Ctrl+C in the console window.


Restore TLS version and cipher support configuration
 
252 

An NFM-P system upgrade does not preserve your changes to the system support for specific TLS versions and ciphers.

If the system had customized TLS settings before the upgrade, see the NSP System Administrator Guide for information about how to restore the TLS version and cipher support settings.

Note: TLS 1.0 and 1.1 are disabled by default after an upgrade. If either version is enabled before an NFM-P system upgrade and required after the upgrade, you must re-enable the version support after the upgrade.


Configure and enable firewalls
 
253 

If you intend to use any firewalls between the NFM-P components, and the firewalls are disabled, configure and enable each firewall.

Perform one of the following.

  1. Configure each external firewall to allow the required traffic using the port assignments in the NSP Planning Guide, and enable the firewall.

  2. Configure and enable firewalld on each component station, as required.

    1. Use an NFM-P template to create the firewalld rules for the component, as described in the NSP Planning Guide.

    2. Log in to the station as the root user.

    3. Open a console window.

    4. Enter the following:

      systemctl enable firewalld ↵

    5. Enter the following:

      systemctl start firewalld ↵

    6. Close the console window.

End of steps