Obtain software

Download the following installation files to an empty local directory on a station that is reachable by each auxiliary database station in the cluster:

• nspos-auxdb-R.r.p-rel.v.rpm

• VerticaSw_PreInstall.sh

• nspos-jre-R.r.p-rel.v.rpm

• vertica-R.r.p-rel.tar

where

R.r.p is the NSP release identifier, in the form MAJOR.minor.patch

v is a version number

Commission new stations, if required

If you are deploying the auxiliary database on one or more new stations, perform the following steps.

For information about deploying the RHEL OS using an NSP OEM disk image, see NSP disk-image deployment.

Note: The IP addresses on the interfaces of a new auxiliary database station must match the addresses on the station that it replaces.

1. Commission each station according to the platform specifications in this guide and in the NSP Planning Guide.

2. Perform To apply the RHEL 8 swappiness workaround on the station.

Back up database

CAUTION Data Loss

If you specify a backup location on the database data partition, data loss or corruption may occur.

The auxiliary database backup location must be an absolute path on a partition other than the database data partition.

If you are upgrading a standalone auxiliary database, or the standby cluster in a redundant auxiliary database, back up the auxiliary database.

Note: The backup location requires 20% more space than the database data consumes.

Note: If the backup location is remote, a 1 Gb/s link to the location is required; if achievable, a higher-capacity link is recommended.

For auxiliary database backup information, see the NSP System Administrator Guide for the installed release.

Stop cluster

Log in as the root user on an auxiliary database station in the cluster that is being upgraded.

Open a console window.

Enter the following:

# cd /opt/nsp/nfmp/auxdb/install/bin ↵

Enter the following to stop the auxiliary database:

# ./auxdbAdmin.sh stop ↵

Enter the following to display the auxiliary database status:

# ./auxdbAdmin.sh status ↵

Information like the following is displayed:

Database status

 Node       | Host          | State | Version | DB

------------+---------------+-------+---------+-------

 node_1 | internal_IP_1 | STATE | version | db_name

 node_2 | internal_IP_2 | STATE | version | db_name

.

.

.

 node_n | internal_IP_n | STATE | version | db_name

      Output captured in log_file

The cluster is stopped when each STATE entry reads DOWN.

Repeat Step 8 periodically until the cluster is stopped.

Note: You must not proceed to the next step until the cluster is stopped.

Prepare all stations for upgrade

Perform Step 12 to Step 28 on each auxiliary database station in the cluster that is being upgraded.

Go to Step 29.

Prepare individual station for upgrade

Log into the auxiliary database station as the root user.

Open a console window.

Enter the following sequence of commands to stop the auxiliary database services:

# systemctl stop nfmp-auxdb.service ↵

# systemctl stop vertica_agent.service ↵

# systemctl stop verticad.service ↵

Enter the following sequence of commands to disable the database services:

# systemctl disable nfmp-auxdb.service ↵

# systemctl disable nfmp-auxdbproxy.service ↵

# systemctl disable vertica_agent.service ↵

# systemctl disable verticad.service ↵

Transfer the downloaded installation files to an empty directory on the station.

Note: You must ensure that the directory is empty.

Note: In subsequent steps, the directory is called the software directory.

Navigate to the software directory.

Note: The directory must contain only the installation files.

Enter the following:

# chmod +x * ↵

Enter the following:

# ./VerticaSw_PreInstall.sh ↵

Information like the following is displayed:

Logging Vertica pre install checks to log_file

INFO: About to remove proxy parameters set by a previous run of this script from /etc/profile.d/proxy.sh

INFO: Completed removing proxy parameters set by a previous run of this script from /etc/profile.d/proxy.sh

INFO: About to set proxy parameters in /etc/profile.d/proxy.sh...

INFO: Completed setting proxy parameters in /etc/profile.d/proxy.sh...

INFO: About to remove kernel parameters set by a previous run of this script from /etc/sysctl.conf

INFO: Completed removing kernel parameters set by a previous run of this script from /etc/sysctl.conf

INFO: About to set kernel parameters in /etc/sysctl.conf...

INFO: Completed setting kernel parameters in /etc/sysctl.conf...

INFO: About to change the current values of the kernel parameters

INFO: Completed changing the current values of the kernel parameters

INFO: About to remove ulimit parameters set by a previous run of this script from /etc/security/limits.conf

INFO: Completed removing ulimit parameters set by a previous run of this script from /etc/security/limits.conf

INFO: About to set ulimit parameters in /etc/security/limits.conf...

INFO: Completed setting ulimit parameters in /etc/security/limits.conf...

Checking Vertica DBA group samauxdb...

WARNING: Vertica DBA group with the specified name already exists locally.

Checking Vertica user samauxdb...

WARNING: Vertica user with the specified name already exists locally.

Changing ownership of the directory /opt/nsp/nfmp/auxdb/install to samauxdb:samauxdb.

Adding samauxdb to sudoers file.

Changing ownership of /opt/nsp/nfmp/auxdb files.

INFO: About to remove commands set by a previous run of this script from /etc/rc.d/rc.local

INFO: Completed removing commands set by a previous run of this script from /etc/rc.d/rc.local

INFO: About to add setting to /etc/rc.d/rc.local...

INFO: Completed adding setting to /etc/rc.d/rc.local...

Enter the following to reboot the station:

# systemctl reboot ↵

The station reboots.

When the reboot is complete, log in to the station as the root user.

Open a console window.

Enter the following:

# cd /opt/nsp/nfmp/auxdb/install/bin ↵

Enter the following to display the auxiliary database status:

# ./auxdbAdmin.sh status ↵

Information like the following is displayed:

Database status

 Node       | Host          | State | Version | DB

------------+---------------+-------+---------+-------

 node_1 | internal_IP_1 | STATE | version | db_name

 node_2 | internal_IP_2 | STATE | version | db_name

.

.

.

 node_n | internal_IP_n | STATE | version | db_name

      Output captured in log_file

if any STATE entry is not DOWN, perform the following steps.

1. Enter the following to stop the auxiliary database:

   # ./auxdbAdmin.sh stop ↵

2. Repeat Step 24 periodically until each STATE entry reads DOWN.

   Note: You must not proceed to the next step until each STATE entry reads DOWN.

Navigate to the software directory.

Enter the following:

# yum install nspos-*.rpm ↵

The yum utility resolves any package dependencies, and displays the following prompt for each package:

Total size: nn G

Installed size: nn G 

Is this ok [y/d/N]: 

Enter y. The following and the installation status are displayed as each package is installed:

Downloading Packages:

Running transaction check

Transaction check succeeded.

Running transaction test

Transaction test succeeded.

Running transaction check

The package installation is complete when the following is displayed:

Complete!

Upgrade database

Log in as the root user on an auxiliary database station in the cluster that is being upgraded.

Open a console window.

Enter the following:

# cd /opt/nsp/nfmp/auxdb/install/bin ↵

Enter the following:

# ./auxdbAdmin.sh upgrade tar_file ↵

where tar_file is the absolute path and filename of the vertica-R.r.p-rel.tar file in the software directory

The following prompt is displayed:

Updating Vertica - Please perform a backup before proceeding with this option

Do you want to proceed (YES/NO)?

Enter YES ↵.

The following prompt is displayed:

Please enter auxiliary database dba password [if you are doing initial setup for auxiliary database, press enter]:

Enter the dba password.

The following prompt is displayed:

Please verify auxiliary database dba password:

Enter the dba password again.

The upgrade begins, and operational messages are displayed.

The upgrade is complete when the following is displayed:

Database database_name started successfully

  Output captured in log_file

Enter the following to display the auxiliary database status:

# ./auxdbAdmin.sh status ↵

Information like the following is displayed:

Database status

 Node       | Host          | State | Version | DB

------------+---------------+-------+---------+-------

 node_1 | internal_IP_1 | STATE | version | db_name

 node_2 | internal_IP_2 | STATE | version | db_name

.

.

.

 node_n | internal_IP_n | STATE | version | db_name

      Output captured in log_file

The cluster is running when each STATE entry reads UP.

Repeat Step 36 periodically until the cluster is running.

Note: You must not proceed to the next step until the cluster is running.

Back up database for migration to new OS version

If no backup has previously been performed on the cluster, for example, if the cluster has always had the standby role, perform the following steps.

1. Open a console window on a station in the current cluster.

2. Copy the following file from a station in the peer cluster to the same directory on the current station:

   backup path/samAuxDbBackup_restore.conf

   where backup_path is the backup location specified in Step 3

3. Enter the following:

   # chown samauxdb:samauxdb backup_path/samAuxDbBackup_restore.conf ↵

4. Enter the following:

   # ./auxdbAdmin.sh status ↵

   Information like the following is displayed:

   Database status

    Node       | Host          | State | Version | DB

   ------------+---------------+-------+---------+-------

    node_1 | internal_IP_1 | STATE | version | db_name

    node_2 | internal_IP_2 | STATE | version | db_name

   .

   .

   .

    node_n | internal_IP_n | STATE | version | db_name

         Output captured in log_file

5. Open the following file using a plain-text editor such as vi:

   backup_path/samAuxDbBackup_restore.conf

6. Locate the section that begins with the following:

   [mapping]

7. Replace the primary cluster node values in the section with the standby node values displayed in substep 4, as shown in the following example for a three-node cluster:

   node_1 = [internal_IP_1]:backup_path

   node_2 = [internal_IP_2]:backup_path

   node_3 = [internal_IP_3]:backup_path

8. Save and close the file.

You must back up the auxiliary database data and configuration information to prepare for the migration to the new OS version.

Note: The operation may take considerable time.

Perform the following steps.

1. Enter the following:

   # ./auxdbAdmin.sh backup /path/samAuxDbBackup_restore.conf ↵

   where path is the backup file path specified in Step 3

   The following prompt is displayed:

   Please enter auxiliary database dba password [if you are doing initial setup for auxiliary database, press enter]:

2. Enter the dba password.

   The backup operation begins, and messages like the following are displayed.

   Copying backup config file to /tmp/auxdbadmin-backup-ID

   Backup snapshot name - AuxDbBackUpID_auxdbAdmin

   Starting auxiliary database backup...

   The backup is complete when the following is displayed:

   Output captured in 

   /opt/nsp/nfmp/auxdb/install/log/auxdbAdmin.sh.timestamp.log

3. Copy the following file from one auxiliary database station to a secure location that is unaffected by the upgrade activity:

   /opt/nsp/nfmp/auxdb/install/config/install.config

Recommission stations, if required

If you are reusing any auxiliary database stations, recommission each station according to the platform specifications in this guide and in the NSP Planning Guide.

For information about deploying the RHEL OS using an NSP OEM disk image, see NSP disk-image deployment.

Note: You must reformat the following disk partitions:

• root

• /opt/nsp/nfmp/auxdb/data

Install new software, restore database

Perform To prepare a station for auxiliary database installation on each auxiliary database station.

Log in as the root user on any auxiliary database station in the cluster.

Copy the install.config file saved in Step 39, substep 3 to the following directory:

/opt/nsp/nfmp/auxdb/install/config

Enter the following:

# /opt/nsp/nfmp/auxdb/install/bin/auxdbAdmin.sh install ↵

The script sequentially prompts for the root user password of each auxiliary database station.

Enter the required password at each prompt. The script installs the software on the station.

When the script execution is complete, if you are deploying a geo-redundant auxiliary database, perform the following steps on each auxiliary database station in the current cluster.

1. Log in to the station as the root user.

2. Open a console window.

3. Enter the following:

   bash$ su - samauxdb ↵

4. Enter the following for each station in the geo-redundant cluster:

   bash$ ssh-copy-id station_IP ↵

   where station_IP is the IP address of a station in the geo-redundant cluster

Stop the auxiliary database.

1. Enter the following on any station in the cluster:

   # cd /opt/nsp/nfmp/auxdb/install/bin ↵

2. Enter the following to stop the auxiliary database:

   # ./auxdbAdmin.sh stop ↵

3. Enter the following to display the auxiliary database status:

   # ./auxdbAdmin.sh status ↵

   Information like the following is displayed:

   Database status

    Node       | Host          | State | Version | DB

   ------------+---------------+-------+---------+-------

    node_1 | internal_IP_1 | STATE | version | db_name

    node_2 | internal_IP_2 | STATE | version | db_name

   .

   .

   .

    node_n | internal_IP_n | STATE | version | db_name

         Output captured in log_file

   The cluster is stopped when each STATE entry reads DOWN.

4. Repeat substep 3 periodically until the cluster is stopped.

   Note: You must not proceed to the next step until the cluster is stopped.

Restore the database backup created in Step 39; see the auxiliary database restore procedure in the NSP System Administrator Guide for information.

Update database schema

Update the NFM-P database schema.

Note: The schema update may take considerable time.

1. Log in as the nsp user on the NFM-P main server.

2. Open a console window.

3. Enter the following:

   bash$ cd /opt/nsp/nfmp/server/nms/bin ↵

4. Enter the following:

   bash$ ./nmsserver.bash upgradeAuxDbSchema ↵

   The following prompt is displayed:

   Auxiliary database clusters:

   1: IP_a,IP_b,IP_c

   2: IP_x,IP_y,IP_z

   Select auxiliary database to upgrade:

5. Enter the number that corresponds to the cluster you are upgrading.

   The following messages and prompt are displayed:

   WARNING: About to upgrade samdb schema on the auxiliary database cluster [IP_a,IP_b,IP_c].

   It is recommended that a database backup is performed before proceeding.

   Type "YES" to continue

6. Enter YES.

   The following prompt is displayed:

   Please enter the auxiliary database port [5433]:

7. Enter the auxiliary database port number; press Enter to accept the default of 5433.

   The following prompt is displayed:

   Please enter the auxiliary database user password:

8. Enter the required password.

   The following messages are displayed as the upgrade begins:

   INFO: Database upgrade can take a very long time on large databases.

   INFO: logs are stored under /opt/nsp/nfmp/server/nms/log/auxdb. Check the logs for progress.

   INFO: Node Name[v_samdb_node0001]->IP[IP_address]->Status[UP]

   INFO: About to perform upgrade

Enable required database services and proxies

If the NFM-P is in a shared-mode NSP system, perform the following steps.

1. Log on as the root user on the NSP cluster host.

2. Open a console window.

3. Enter the following:

   # cd /opt ↵

4. Enter the following:

   # sftp root@deployer_IP ↵

   where deployer_IP is the NSP deployer host IP address

   The prompt changes to sftp>.

5. Enter the following:

   sftp> cd /opt/nsp/NSP-CN-DEP-release-ID/NSP-CN-release-ID/tools/database ↵

6. Enter the following:

   sftp> get sync-auxdb-password.bash ↵

7. Enter the following:

   sftp> quit ↵

8. Enter the following:

   # chmod 777 sync-auxdb-password.bash ↵

9. Enter the following:

   # ./sync-auxdb-password.bash ↵

10. If the command in substep 9 succeeds, output like the following is displayed:

    timestamp: Synchronizing password for Auxiliary DB Output...

    timestamp: deployment.apps/tlm-vertica-output scaled

    timestamp: secret/tlm-vertica-output patched

    timestamp: deployment.apps/tlm-vertica-output scaled

    timestamp: Synchronization completed.

11. If the command output is not as expected, the NFM-P initialization may not be complete; wait 30 minutes, and then return to substep 9. Several attempts may be required.

Perform the following steps on each station in the auxiliary database cluster.

1. Log in as the root user.

2. Open a console window.

3. Enter the following sequence of commands to enable the database services:

   systemctl enable nspos-auxdb.service

   systemctl enable nspos-auxdbproxy.service

   systemctl enable vertica_agent.service

   systemctl enable verticad.service

If you are upgrading the standby auxiliary database cluster in a redundant deployment, go to Step 54.

Enter the following on each station in the auxiliary database cluster to start the database proxy:

# systemctl start nspos-auxdbproxy.service ↵

Close the open console windows.

End of steps