Prepare for data recovery

Retrieve the required nsp-tomcat database backup.

1. If the required backup is stored in the nsp-backup-storage-0 pod, perform the following steps.

   1. Log in as the root or NSP admin user on the NSP cluster host in the cluster that has the database backup.

   2. Open a console window.

   3. Enter the following:

      # kubectl exec nsp-backup-storage-0 -- ls /tmp/backups/nsp-tomcat/ ↵

      The backup files are listed.

   4. Record the name of the most recent backup.

   5. Enter the following:

      # kubectl cp nsp-backup-storage-0:/tmp/backups/nsp-tomcat/backup_file /tmp/backup_file ↵

      where backup_file is the recorded backup file name

      The backup file is copied to the temp directory on the NSP cluster host.

2. If the required backup is from the nsp-tomcat startup:

   1. Log in as the root or NSP admin user on the NSP cluster host in the cluster that has the database backup.

   2. Open a console window.

   3. Enter the following:

      # kubectl exec $(kubectl get pods | grep nsp-tomcat | egrep -v "haproxy|backup" | tail -1 | awk '{print $1}') -c nsp-tomcat -- ls /opt/nsp/server/tomcat/work/replicationArchives/ ↵

      The backup files are listed.

   4. Record the name of the most recent backup.

   5. Enter the following:

      # kubectl cp $(kubectl get pods | grep nsp-tomcat | egrep -v "haproxy|backup" | tail -1 | awk '{print $1}'):/opt/nsp/server/tomcat/work/replicationArchives/backup_file /tmp/backup_file ↵

      where backup_file is the recorded backup file name

      The backup file is copied to the temp directory on the NSP cluster host.

Copy the backup file to the NSP deployer host in each data center.

1. Enter the following:

   # scp /tmp/backup_file address:/tmp ↵

   where address is the address of the local NSP deployer host

   The backup file is copied to the /tmp directory on the NSP deployer host.

2. Enter the following:

   # scp /tmp/backup_file address:/tmp ↵

   where address is the address of the NSP deployer host in the other data center

   The backup file is copied to the /tmp directory on the NSP deployer host in the other data center.

Disable the auto-switchover function to ensure that no switchover occurs during the database restoration.

Enter the following as the root or NSP admin user on the NSP cluster host in each data center:

# kubectl exec -it $(kubectl get pods | awk '/nspos-asm/ {print $1;exit}') -c nspos-asm-app -- /opt/nsp/os/asm/bin/asmctl disableAutoFailover ↵

Enter the following as the root or NSP admin user on the NSP cluster host in either data center:

# helm list | grep nsp-tomcat | awk -F' ' '{ print $10 }' ↵

The nsp-tomcat version is displayed.

Record the nsp-tomcat version.

Perform Step 8 to Step 21 in each data center.

Note: You must perform the steps in the primary data center first.

Go to Step 22.

Restore individual cluster

Enter the following on the NSP cluster host to uninstall nsp-tomcat:

# helm uninstall -n nsp-psa-restricted nsp-tomcat ↵

The nsp-tomcat uninstallation begins.

Enter the following command block to monitor the uninstallation:

while [ 0 -lt `kubectl get pvc --all-namespaces | grep nsp-tomcat | wc -l` ] || [ 0 -lt `kubectl get pv --all-namespaces | grep nsp-tomcat | wc -l` ]

do

sleep 2

echo "Still there..."

done

The uninstallation is complete when the command prompt is displayed.

When the uninstallation is complete, enter the following to list the cluster nodes that can host nsp-tomcat:

# kubectl get nodes --show-labels -o wide | grep nsp-sdn=true ↵

The nodes are listed; the following is a truncated output example for one node:

node_name   Ready    control-plane   nnd   vx.yy.zz   IP_address

Enter the following command block as the root user on each node listed in Step 10 to delete the nsp-tomcat volume data directory content:

Note: The cluster-state folder may already be empty.

rm -rf /opt/nsp/volumes/nsp-tomcat/data/databases

rm -rf /opt/nsp/volumes/nsp-tomcat/data/cluster-state

rm -rf /opt/nsp/volumes/nsp-tomcat/data/transactions

Enter the following on the NSP cluster host to install nsp-tomcat:

# helm upgrade nsp-tomcat --install oci://registry.nsp.nokia.local/nsp/charts/cn-nsp-tomcat --namespace nsp-psa-restricted --version version --timeout 300s -f /opt/nsp/config/helm/values/sdn/nsp-tomcat/values.yaml -f /opt/nsp/config/helm/values/sdn/nsp-tomcat/restore.yaml ↵

where version is the nsp-tomcat version recorded in Step 4

nsp-tomcat is installed.

Restore the nsp-tomcat database.

Enter the following as the root or NSP admin user on the NSP deployer host:

# /opt/nsp/NSP-CN-DEP-release-ID/NSP-CN-release-ID/tools/database/

nspos-db-restore-k8s.sh nsp-tomcat /tmp/backup_file ↵

You are prompted to restore the database.

Respond to the prompt.

The nsp-tomcat database is restored from the backup file.

Enter the following on the NSP cluster host to uninstall nsp-tomcat:

# helm uninstall -n nsp-psa-restricted nsp-tomcat ↵

The nsp-tomcat uninstallation begins.

Note: You must not proceed to the next step until the uninstallation is complete.

Enter the following command block to monitor the uninstallation:

while [ 0 -lt `kubectl get pvc --all-namespaces | grep nsp-tomcat | wc -l` ] || [ 0 -lt `kubectl get pv --all-namespaces | grep nsp-tomcat | wc -l` ]

do

sleep 2

echo "Still there..."

done

The uninstallation is complete when the command prompt is displayed.

When the uninstallation is complete, enter the following to install nsp-tomcat:

# helm upgrade nsp-tomcat --install oci://registry.nsp.nokia.local/nsp/charts/cn-nsp-tomcat --namespace nsp-psa-restricted --version version --timeout 300s -f /opt/nsp/config/helm/values/sdn/nsp-tomcat/values.yaml ↵

where version is the nsp-tomcat version recorded in Step 4

nsp-tomcat is installed.

Enter the following:

# kubectl get pods -A ↵

The pods are listed.

If the nsp-tomcat pod is not running, repeat the command in Step 18.

Note: You must not proceed to the next step until nsp-tomcat is running.

Sign in to the NSP as the admin user.

Verify that you can open Path Control.

You must not proceed to the next step until Path Control is available.

Complete standby cluster configuration

Enter the following on the NSP cluster host in the standby data center to install nsp-tomcat:

# helm upgrade nsp-tomcat --install oci://registry.nsp.nokia.local/nsp/charts/cn-nsp-tomcat --namespace nsp-psa-restricted --version version --timeout 300s -f /opt/nsp/config/helm/values/sdn/nsp-tomcat/values.yaml ↵

where version is the nsp-tomcat version recorded in Step 4

nsp-tomcat is installed.

Enable the auto-switchover function; enter the following as the root or NSP admin user on the NSP cluster host in each data center:

# kubectl exec -it $(kubectl get pods | awk '/nspos-asm/ {print $1;exit}') -c nspos-asm-app -- /opt/nsp/os/asm/bin/asmctl enableAutoFailover ↵

Monitor the following log files in the nsp-tomcat pod to ensure that the database starts successfully:

• primary NSP cluster host—/opt/nsp/server/tomcat/work/logs/debug.log

• standby NSP cluster host—/opt/nsp/server/replication/logs/debug.log

Close the open console windows.

End of steps