To upgrade a standalone auxiliary database

Description

The following procedure describes how to upgrade a standalone NSP auxiliary database.

Note: You require the following user privileges on each station:

Note: Ensure that you record the information that you specify, for example, directory names, passwords, and IP addresses.

Note: The following RHEL CLI prompts in command lines denote the active user, and are not to be included in typed commands

Steps
Obtain software
 
1 

Download the following installation files to an empty local directory on a station that is reachable by each station in the auxiliary database cluster:

  • nspos-auxdb-R.r.p-rel.v.rpm

  • VerticaSw_PreInstall.sh

  • nspos-jre-R.r.p-rel.v.rpm

  • vertica-R.r.p-rel.tar

where

R.r.p is the NSP release identifier, in the form MAJOR.minor.patch

v is a version number

Back up database
 
2 
CAUTION 

CAUTION

Data Loss

If you specify a backup location on the database data partition, data loss or corruption may occur.

The auxiliary database backup location must be an absolute path on a partition other than the database data partition.

Back up the auxiliary database as described in the NSP System Administrator Guide for the installed release.

Note: The backup location requires 20% more space than the database data consumes.

Note: If the backup location is remote, a 1 Gb/s link to the location is required; if achievable, a higher-capacity link is recommended.

Enable auxiliary database maintenance mode
 
3 

Perform the following steps.

  1. Log in as the root or NSP admin user on the NSP cluster host .

  2. Enter the following:

    # kubectl patch configmap/nspos-auxdb-agent-overrides -n namespace --type=merge -p '{"data":{"nspos-auxdb-agent-overrides.json":"{\"auxDbAgent\":{\"config\":{\"maintenance-mode\":true}}}"}}' ↵

    where namespace is the nspos-auxdb-agent namespace

  3. Enter the following to restart the nspos-auxdb-agent pod:

    # kubectl delete -n namespace pod `kubectl describe -n namespace pods | grep -P ^^Name: | grep -oP nspos-auxdb-agent[-a-zA-Z0-9]+` ↵

    Do not perform the next step until the nspos-auxdb-agent pod is up and running.

  4. Issue the following RESTCONF API call against the primary NSP cluster to verify that the agent is in maintenance mode:

    Note: In order to issue a RESTCONF API call, you require a token; see this tutorial on the Network Developer Portal for information.

    GET https://address/restconf/data/auxdb:auxdb-agent

    where address is the advertised address of the primary NSP cluster

    The call returns information like the following:

    {

        "auxdb-agent": {

            "name": "nspos-auxdb-agent",

            "application-mode": "MAINTENANCE",

            "copy-cluster": {

                "source-cluster": "",

                "target-cluster": "",

                "time-started": "",

                "status": "UNKNOWN"

            }

        }

    }

    The agent is in maintenance mode if the application-mode is MAINTENANCE, as shown in the example.

Stop and disable database proxies
 
4 

Perform the following to stop and disable the auxiliary database proxy services.

  1. Log in as the root user on each auxiliary database station in the cluster.

  2. Perform one of the following:

    1. If you are upgrading from Release 23.11 or later, enter the following:

      # systemctl stop nspos-auxdbproxy.service ↵

      # systemctl disable nspos-auxdbproxy.service ↵

    2. If you are upgrading from Release 23.8 or earlier, enter the following:

      # systemctl stop nfmp-auxdbproxy.service ↵

      # systemctl disable nfmp-auxdbproxy.service ↵

Stop auxiliary database
 
5 

Log in as the root user on an auxiliary database station.

6 

Open a console window.

7 

Enter the following:

# cd /opt/nsp/nfmp/auxdb/install/bin ↵

8 

Enter the following to block external access to the auxiliary database ports:

# ./auxdbAdmin.sh shieldsUp ↵

9 

Enter the following to stop the auxiliary database:

# ./auxdbAdmin.sh stop ↵

10 

Enter the following to display the auxiliary database status:

# ./auxdbAdmin.sh status ↵

Information like the following is displayed:

Database status

 Node       | Host          | State | Version | DB

------------+---------------+-------+---------+-------

 node_1     | internal_IP_1 | STATE | version | db_name

 node_2     | internal_IP_2 | STATE | version | db_name

.

.

.

 node_n     | internal_IP_n | STATE | version | db_name

      Output captured in log_file

The cluster is stopped when each STATE entry reads DOWN.

Note: If the cluster is not stopped, enter the following to force the auxiliary database to stop.

# ./auxdbAdmin.sh force_stop ↵

11 

Repeat Step 10 periodically until the cluster is stopped.

Note: You must not proceed to the next step until the cluster is stopped.

Prepare all stations for upgrade
 
12 

Perform Step 14 to Step 31 on each auxiliary database station.

13 

Go to Step 32.

Prepare individual station for upgrade
 
14 

If the auxiliary database station is deployed in a VM created using an NSP RHEL OS disk image, perform To apply a RHEL update to an NSP image-based OS.

15 

Log in as the root user on the auxiliary database station.

16 

Open a console window.

17 

Perform one of the following to stop the database services:

  1. If you are upgrading from Release 23.11 or later, enter the following sequence of commands:

    systemctl stop nspos-auxdb.service

    systemctl disable nspos-auxdb.service

  2. If you are upgrading from Release 23.8 or earlier, enter the following sequence of commands:

    systemctl stop nfmp-auxdb.service

    systemctl disable nfmp-auxdb.service

18 

Transfer the downloaded installation files to an empty directory on the station.

Note: You must ensure that the directory is empty.

Note: In subsequent steps, the directory is called the NSP software directory.

19 

Navigate to the NSP software directory.

Note: The directory must contain only the installation files.

20 

Enter the following:

# chmod +x * ↵

21 

Enter the following:

# ./VerticaSw_PreInstall.sh ↵

Messages like the following are displayed:

Logging Vertica pre install checks to log_file

INFO: About to remove proxy parameters set by a previous run of 

            this script from    /etc/profile.d/proxy.sh

INFO: Completed removing proxy parameters set by a previous run of 

            this script from    /etc/profile.d/proxy.sh

INFO: About to set proxy parameters in /etc/profile.d/proxy.sh...

INFO: Completed setting proxy parameters in /etc/profile.d/proxy.sh...

INFO: Found pre-existing [/opt/nsp/nfmp/auxdb/install/config/install.config]

      file on this machine. This file will be validated.

INFO: About to remove kernel parameters set by a previous run of 

      this script from /etc/sysctl.conf

INFO: Completed removing kernel parameters set by a previous run of 

      this script from /etc/sysctl.conf

INFO: About to set kernel parameters in /etc/sysctl.conf...

INFO: Completed setting kernel parameters in /etc/sysctl.conf...

INFO: About to change the current values of the kernel parameters

INFO: Completed changing the current values of the kernel parameters

INFO: About to remove ulimit parameters set by a previous run of 

      this script from  /etc/security/limits.conf

INFO: Completed removing ulimit parameters set by a previous run of 

      this script from  /etc/security/limits.conf

INFO: About to set ulimit parameters in /etc/security/limits.conf...

INFO: Completed setting ulimit parameters in /etc/security/limits.conf...

INFO: Removing /var/log/wtmp entry from /etc/logrotate.conf

INFO: Adding /etc/logrotate.d/wtmp

Checking user group nsp... 

WARNING: user group with the specified name already exists locally.

Checking user nsp... 

INFO: user with the specified name already exists locally. Setting shell to /usr/sbin/nologin.

Checking Vertica user group samauxdb... 

WARNING: Vertica user group with the specified name already exists locally.

Checking Vertica user samauxdb... 

WARNING: Vertica user with the specified name already exists locally.

Moving logfile from /tmp ...

     ... to /opt/nsp/nfmp/auxdb/install/log

Changing ownership of the directory /opt/nsp/nfmp/auxdb/install to samauxdb:samauxdb.

Removing group write and world permissions from the directory /opt/nsp/nfmp/auxdb/install.

Changing ownership of /opt/nsp/nfmp/auxdb files.

Set virtio device queue scheduler for /dev/vda: [none] mq-deadline kyber bfq 

INFO: Updating auxiliary database prep script.

************************************************************************ Changes were made that require a restart.                            * Please restart this host before proceeding with Vertica installation.***********************************************************************
22 

Enter the following to reboot the station:

# systemctl reboot ↵

The station reboots.

23 

When the reboot is complete, log in as the root user on the station.

24 

Open a console window.

25 

On one of the auxiliary database stations in the cluster, enter the following:

# cd /opt/nsp/nfmp/auxdb/install/bin ↵

26 

Enter the following to block external access to the auxiliary database ports:

# ./auxdbAdmin.sh shieldsUp ↵

27 

Enter the following to display the auxiliary database status:

# ./auxdbAdmin.sh status ↵

Information like the following is displayed:

Database status

 Node       | Host          | State | Version | DB

------------+---------------+-------+---------+-------

 node_1     | internal_IP_1 | STATE | version | db_name

 node_2     | internal_IP_2 | STATE | version | db_name

.

.

.

 node_n     | internal_IP_n | STATE | version | db_name

      Output captured in log_file

28 

if any STATE entry is not DOWN, perform the following steps.

  1. Enter the following to stop the auxiliary database:

    # ./auxdbAdmin.sh stop ↵

    Note: If the cluster is not stopped, enter the following to force the auxiliary database to stop.

    # ./auxdbAdmin.sh force_stop ↵

  2. Repeat Step 27 periodically until each STATE entry reads DOWN.

    Note: You must not proceed to the next step until each STATE entry reads DOWN.

29 

On each auxiliary database station in the cluster, navigate to the NSP software directory.

30 

Enter the following:

# dnf install nspos-*.rpm ↵

The dnf utility resolves any package dependencies, and displays the following prompt for each package:

Total size: nn G

Is this ok [y/N]: 

31 

Enter y. The following and the installation status are displayed as each package is installed:

Downloading Packages:

Running transaction check

Transaction check succeeded.

Running transaction test

Transaction test succeeded.

Running transaction

The package installation is complete when the following is displayed:

Complete!

Upgrade database
 
32 

Log in as the root user on an auxiliary database station.

33 

Open a console window.

34 

Enter the following:

# cd /opt/nsp/nfmp/auxdb/install/bin ↵

35 

Enter the following to start the upgrade.

# ./auxdbAdmin.sh upgrade tar_file ↵

where tar_file is the absolute path and filename of the vertica-R.r.p-rel.tar file in the NSP software directory.

The following prompt is displayed:

Updating Vertica - Please perform a backup before proceeding with this option

Do you want to proceed (YES/NO)?

36 

Enter YES ↵.

The following prompt is displayed:

Please enter auxiliary database dba password [if you are doing initial setup for auxiliary database, press enter]:

37 

Enter the dba password.

The following prompt is displayed:

Please confirm auxiliary database dba password:

38 

Enter the dba password again.

The upgrade begins, and operational messages are displayed.

The upgrade is complete when the following is displayed:

# Post-upgrade tasks completed.

39 

Enter the following to display the auxiliary database status:

# ./auxdbAdmin.sh status ↵

Information like the following is displayed:

Database status

 Node       | Host          | State | Version | DB

------------+---------------+-------+---------+-------

 node_1     | internal_IP_1 | STATE | version | db_name

 node_2     | internal_IP_2 | STATE | version | db_name

.

.

.

 node_n     | internal_IP_n | STATE | version | db_name

      Output captured in log_file

The cluster is running when each STATE entry reads UP.

40 

Repeat Step 39 periodically until the cluster is running.

Note: You must not proceed to the next step until the cluster is running.

Configure TLS
 
41 

Open the /opt/nsp/nfmp/auxdb/install/config/install.config file using a plain-text editor such as vi.

42 

Edit the following lines in the file to read as shown below:

secure=true

pki_server=address

pki_server_port=80

where address is one of the following in the platform—ingressApplications—ingressController section of the nsp-config.yml file on the local NSP deployer host:

In the internalAddresses subsection, if configured, otherwise, in the clientAddresses subsection:

  • if configured, the advertised value

  • otherwise, the virtualIp value

Note: If an external pki-server is used for generation of NSP certificate artifacts, use the address and port of the external pki-server instead.

43 

Save and close the install.config file.

44 

Enter the following to regenerate the TLS certificates:

# ./auxdbAdmin.sh configureTLS ↵

45 

Enter the following:

# ./auxdbAdmin.sh shieldsDown ↵

Enable required database services and proxies
 
46 

Perform the following steps on each auxiliary database station.

  1. Log in as the root user.

  2. Open a console window.

  3. Enter the following sequence of commands to enable the database services:

    # systemctl enable nspos-auxdb.service ↵

    # systemctl enable nspos-auxdbproxy.service ↵

    # systemctl start nspos-auxdbproxy.service ↵

Disable maintenance mode for auxiliary database agent
 
47 

Perform the following steps.

  1. Log in as the root or NSP admin user on the NSP cluster host.

  2. Enter the following:

    # kubectl patch configmap/nspos-auxdb-agent-overrides -n namespace --type=merge -p '{"data":{"nspos-auxdb-agent-overrides.json":"{\"auxDbAgent\":{\"config\":{\"maintenance-mode\":false}}}"}}' ↵

    where namespace is the nspos-auxdb-agent namespace

  3. Enter the following to restart the nspos-auxdb-agent pod:

    # kubectl delete -n namespace pod `kubectl describe -n namespace pods | grep -P ^^Name: | grep -oP nspos-auxdb-agent[-a-zA-Z0-9]+` ↵

48 

Close the open console windows.

End of steps