Performing a software upgrade

You use the fss-upgrade.sh script to perform the software upgrade steps. The command syntax is as follows:
[root@fss-deployer ~]# ./bin/fss-upgrade.sh
Usage: ./bin/fss-upgrade.sh configure <config-file> - Reads config file and configures accordingly.
 ./bin/fss-upgrade.sh discover - Discovers cluster.
 ./bin/fss-upgrade.sh upgrade - Upgrades FSS application.
 ./bin/fss-upgrade.sh status - Prints status of cluster and FSS Chart information.
 ./bin/fss-upgrade.sh clean - Cleans temporary files.
 ./bin/fss-upgrade.sh [help | -h] - Prints usage.

This script updates the entire Fabric Services System application using the new Helm charts.

Ensure that you have completed the steps described in Preparing for software upgrade. You execute the steps in this procedure while logged in to the new deployer VM that you created in Preparing for software upgrade.
  1. Update the configuration.
    In the following example, the input configuration file is customer-input.json.
    [root@fss-deployer ~]# ./bin/fss-upgrade.sh configure customer-input.json
        Timesync service is running on 192.168.59.195  Time difference is 0 seconds
        Timesync service is running on 192.168.59.129  Time difference is 0 seconds
        Timesync service is running on 192.168.59.127  Time difference is 0 seconds
      Maximum time difference between nodes 0 seconds
      Successfully configured. Please run /root/bin/fss-upgrade.sh upgrade
    Note: Do not execute the fss-upgrade.sh upgrade command as indicated in the output. Continue with Step 2.
  2. Discover the cluster.
    [root@fss-deployer ~]# ./bin/fss-upgrade.sh discover
    Trying to discover 192.168.59.195
    Trying to discover 192.168.59.129
    Trying to discover 192.168.59.127
    NAME                      STATUS   ROLES                  AGE   VERSION
    fss-compute-customer-01   Ready    control-plane,master   12h   v1.23.1
    fss-compute-customer-02   Ready    control-plane,master   12h   v1.23.1
    fss-compute-customer-03   Ready    control-plane,master   12h   v1.23.1
    FSS release discovered fss-FSS_22_8_B1-charts-v22.8.1-108
  3. Upgrade the Fabric Services System application.
    [root@fss-deployer ~]# /root/bin/fss-upgrade.sh upgrade
    NAME                      STATUS   ROLES                  AGE   VERSION
    fss-compute-customer-01   Ready    control-plane,master   12h   v1.23.1
    fss-compute-customer-02   Ready    control-plane,master   12h   v1.23.1
    fss-compute-customer-03   Ready    control-plane,master   12h   v1.23.1
    FSS will be upgraded from fss-FSS_22_8_B1-charts-v22.8.1-108 to fss-FSS_22_8_B1-charts v22.8.2-6 : Are you sure [YyNn]? y
    Upgrade in progress...
    W1027 18:06:09.880444    7001 warnings.go:67] policy/v1beta1 PodDisruptionBudget is deprecated in v1.21+, unavailable in v1.25+; use policy/v1 PodDisruptionBudget
    W1027 18:06:09.888237    7001 warnings.go:67] policy/v1beta1 PodDisruptionBudget is deprecated in v1.21+, unavailable in v1.25+; use policy/v1 PodDisruptionBudget
    W1027 18:06:09.905507    7001 warnings.go:67] policy/v1beta1 PodDisruptionBudget is deprecated in v1.21+, unavailable in v1.25+; use policy/v1 PodDisruptionBudget
    W1027 18:06:09.910756    7001 warnings.go:67] policy/v1beta1 PodDisruptionBudget is deprecated in v1.21+, unavailable in v1.25+; use policy/v1 PodDisruptionBudget
    W1027 18:06:09.918327    7001 warnings.go:67] policy/v1beta1 PodDisruptionBudget is deprecated in v1.21+, unavailable in v1.25+; use policy/v1 PodDisruptionBudget
    W1027 18:06:09.927503    7001 warnings.go:67] policy/v1beta1 PodDisruptionBudget is deprecated in v1.21+, unavailable in v1.25+; use policy/v1 PodDisruptionBudget
    W1027 18:06:09.932171    7001 warnings.go:67] policy/v1beta1 PodDisruptionBudget is deprecated in v1.21+, unavailable in v1.25+; use policy/v1 PodDisruptionBudget
    W1027 18:06:09.938165    7001 warnings.go:67] policy/v1beta1 PodDisruptionBudget is deprecated in v1.21+, unavailable in v1.25+; use policy/v1 PodDisruptionBudget
    W1027 18:06:09.945057    7001 warnings.go:67] policy/v1beta1 PodDisruptionBudget is deprecated in v1.21+, unavailable in v1.25+; use policy/v1 PodDisruptionBudget
    Release "prod" has been upgraded. Happy Helming!
    NAME: prod
    LAST DEPLOYED: Thu Oct 27 18:06:07 2022
    NAMESPACE: default
    STATUS: deployed
    REVISION: 2
    NOTES:
    [root@fss-deployer ~]#
    
  4. Verify that the upgrade is successful.
    1. Check the status of the compute nodes.
      The STATUS field should show Ready.
      [root@fss-deployer ~]# ./bin/fss-upgrade.sh status
      NAME                      STATUS   ROLES                  AGE   VERSION
      fss-compute-customer-01   Ready    control-plane,master   12h   v1.23.1
      fss-compute-customer-02   Ready    control-plane,master   12h   v1.23.1
      fss-compute-customer-03   Ready    control-plane,master   12h   v1.23.1
      FSS release discovered: fss-FSS_22_8_B1-charts-v22.8.2-6 ; newer release:  fss-FSS_22_8_B1-charts v22.8.2-6
    2. Display the Helm chart version.
      [root@fss-deployer ~]# export KUBECONFIG=/var/lib/fss/config.fss
      [root@fss-deployer ~]# helm list
      NAME    NAMESPACE       REVISION        UPDATED                                 STATUS          CHART                                   APP VERSION
      prod    default         2               2022-10-27 18:06:07.835241335 +0000 UTC deployed        fss-FSS_22_8_B1-charts-v22.8.2-6        1.0
      
    3. Check the status of system services and processes.
      The upgrade is in progress until the STATUS field for all pods shows Running.
      [root@fss-deployer ~]# kubectl get pods
      NAME                                                   READY   STATUS        RESTARTS      AGE
      prod-cp-kafka-0                                        2/2     Running       2 (11m ago)   13m
      prod-cp-kafka-1                                        2/2     Running       1 (11m ago)   12m
      prod-cp-kafka-2                                        2/2     Running       0             12m
      prod-cp-zookeeper-0                                    2/2     Running       0             13m
      prod-cp-zookeeper-1                                    2/2     Running       0             11m
      prod-cp-zookeeper-2                                    2/2     Running       0             11m
      prod-fss-alarmmgr-6678f4d49c-xz77v                     1/1     Terminating   0             13m
      prod-fss-auth-6875676df7-9b74r                         1/1     Running       0             33s
      prod-fss-auth-79ff5f495c-s4t7m                         0/1     Terminating   0             13m
      prod-fss-catalog-7c5478d8ff-t546z                      0/1     Init:0/1      0             27s
      prod-fss-cfggen-5d8f94f74b-57nd7                       1/1     Running       0             13m
      prod-fss-cfggen-6f5958bdf4-m6s54                       0/1     Init:0/1      0             33s
      prod-fss-cfgsync-5b69b7f854-l9lx5                      1/1     Running       0             13m
      prod-fss-cfgsync-6889dfc69d-t99mg                      0/1     Init:0/1      0             33s
      prod-fss-connect-69d8d49799-96w7k                      0/1     Init:0/1      0             33s
      prod-fss-connect-75df8ff7f9-vwfgj                      1/1     Running       0             13m
      prod-fss-da-69f8676b6d-4kjsn                           0/1     Terminating   0             13m
      prod-fss-da-69f8676b6d-cp5v6                           1/1     Terminating   0             13m
      prod-fss-da-69f8676b6d-fj7db                           1/1     Running       0             13m
      prod-fss-da-69f8676b6d-mnz9b                           1/1     Running       0             13m
      prod-fss-da-69f8676b6d-ph42r                           1/1     Running       0             13m
      prod-fss-da-69f8676b6d-vgflj                           0/1     Terminating   0             13m
      prod-fss-da-69f8676b6d-zg2pb                           1/1     Running       0             13m
      prod-fss-da-69f8676b6d-zt7z6                           1/1     Terminating   0             13m
      prod-fss-da-bf8f8d456-6pmlr                            0/1     Init:0/1      0             33s
      prod-fss-da-bf8f8d456-h9z6j                            1/1     Running       0             33s
      prod-fss-da-bf8f8d456-l9fjh                            0/1     Init:0/1      0             17s
      prod-fss-da-bf8f8d456-lqzmn                            1/1     Running       0             32s
      prod-fss-da-bf8f8d456-ptd7t                            0/1     Init:0/1      0             32s
      prod-fss-da-bf8f8d456-tghht                            0/1     Init:0/1      0             22s
      prod-fss-deviationmgr-acl-6c88f46886-pl4bm             0/1     Init:0/1      0             33s
      prod-fss-deviationmgr-acl-d8949f794-2dl79              1/1     Running       0             13m
      prod-fss-deviationmgr-bfd-68f5d7988b-q8vks             1/1     Running       0             13m
      prod-fss-deviationmgr-bfd-79bff7cd9d-5qst2             0/1     Init:0/1      0             33s
      prod-fss-deviationmgr-interface-769c5d46f-l9wrc        1/1     Running       0             13m
      prod-fss-deviationmgr-interface-77574648bf-zhrwk       0/1     Init:0/1      0             32s
      prod-fss-deviationmgr-netinst-6f455d75d5-gqftv         1/1     Running       0             13m
      prod-fss-deviationmgr-netinst-79d9484657-jsp67         0/1     Init:0/1      0             32s
      prod-fss-deviationmgr-platform-589dd98c4f-l67dp        1/1     Running       0             13m
      prod-fss-deviationmgr-platform-868fc5fdc7-z7xzk        0/1     Init:0/1      0             32s
      prod-fss-deviationmgr-qos-567cbfc7ff-twblt             1/1     Running       0             13m
      prod-fss-deviationmgr-qos-6ffcfcbff5-2f2mj             0/1     Init:0/1      0             32s
      prod-fss-deviationmgr-routingpolicy-7cbcfc858b-rxf7p   1/1     Running       0             13m
      prod-fss-deviationmgr-routingpolicy-c9c5d56db-5zm6r    0/1     Init:0/1      0             31s
      prod-fss-deviationmgr-system-6549bc48ff-2qttp          1/1     Running       0             13m
      prod-fss-deviationmgr-system-6cd48bdb67-42lb8          0/1     Init:0/1      0             31s
      prod-fss-dhcp-c595df9fc-lx4ln                          1/1     Running       0             13m
      prod-fss-filemgr-687fc88b4d-hc8bz                      1/1     Running       0             13m
      prod-fss-filemgr-7546d4945b-xhs2h                      0/1     Init:0/1      0             31s
      prod-fss-imagemgr-5bbddc7995-fj5vm                     0/1     Init:0/1      0             19s
      prod-fss-intentmgr-78fcdf84df-fzmzm                    0/1     Terminating   0             13m
      prod-fss-inventory-7d5544bf6f-d6cm5                    0/1     Init:0/1      0             21s
      prod-fss-labelmgr-7db67c68cf-482kw                     1/1     Running       0             13m
      prod-fss-labelmgr-bdd847459-b5qkm                      0/1     Init:0/1      0             30s
      prod-fss-maintmgr-76d755c8cc-m6njl                     1/1     Terminating   0             13m
      prod-fss-mgmtstack-68b8fc9fdf-6xt7f                    1/1     Running       0             13m
      prod-fss-mgmtstack-7f67b85c97-hj6rd                    0/1     Init:0/1      0             30s
      prod-fss-oper-da-6556b58fc5-29tw6                      1/1     Running       0             13m
      prod-fss-oper-da-6556b58fc5-789nx                      0/1     Terminating   0             13m
      prod-fss-oper-da-6556b58fc5-bpnvg                      1/1     Running       0             13m
      prod-fss-oper-da-6556b58fc5-bvvkc                      1/1     Running       0             13m
      prod-fss-oper-da-6556b58fc5-k7xqz                      1/1     Running       0             13m
      prod-fss-oper-da-6556b58fc5-rlqlv                      0/1     Terminating   0             13m
      prod-fss-oper-da-6556b58fc5-wn2bb                      1/1     Running       0             13m
      prod-fss-oper-da-6556b58fc5-zvkx2                      1/1     Running       0             13m
      prod-fss-oper-da-7cc49477f-fglt4                       0/1     Init:0/1      0             29s
      prod-fss-oper-da-7cc49477f-lmghk                       0/1     Init:0/1      0             30s
      prod-fss-oper-da-7cc49477f-pmkkc                       0/1     Init:0/1      0             28s
      prod-fss-oper-da-7cc49477f-wprbk                       0/1     Init:0/1      0             28s
      prod-fss-oper-topomgr-665bdb7bcc-hwgnm                 0/1     Pending       0             3s
      prod-fss-topomgr-7756dbc5dc-pnfhp                      0/1     Init:0/1      0             29s
      prod-fss-topomgr-77cd566695-5w5mx                      1/1     Running       0             13m
      prod-fss-transaction-5756574f99-clfv5                  1/1     Terminating   0             13m
      prod-fss-version-b565f896-kcktt                        0/1     Terminating   0             13m
      prod-fss-web-5d5bbf4c67-gd6bf                          1/1     Running       0             13m
      prod-fss-web-7967d75595-x7dpb                          0/1     Init:0/1      0             29s
      prod-fss-workloadmgr-77f49c9dbb-gbz74                  0/1     Init:0/1      0             29s
      prod-fss-workloadmgr-c99db6bb6-fft5q                   1/1     Running       0             13m
      prod-fss-ztp-59675c6ffd-pzpxj                          1/1     Running       0             13m
      prod-fss-ztp-644d8c689d-cdn5s                          0/1     Init:0/1      0             29s
      prod-keycloak-0                                        1/1     Running       0             13m
      prod-mongodb-arbiter-0                                 1/1     Running       0             13m
      prod-mongodb-primary-0                                 1/1     Running       0             13m
      prod-mongodb-secondary-0                               1/1     Running       0             13m
      prod-neo4j-core-0                                      1/1     Running       0             13m
      prod-postgresql-0                                      1/1     Running       0             13m
      [root@fss-deployer ~]#
      
      [root@fss-deployer ~]# kubectl get pods
      NAME                                                  READY   STATUS    RESTARTS      AGE
      prod-cp-kafka-0                                       2/2     Running   2 (14m ago)   16m
      prod-cp-kafka-1                                       2/2     Running   1 (14m ago)   15m
      prod-cp-kafka-2                                       2/2     Running   0             15m
      prod-cp-zookeeper-0                                   2/2     Running   0             16m
      prod-cp-zookeeper-1                                   2/2     Running   0             14m
      prod-cp-zookeeper-2                                   2/2     Running   0             14m
      prod-fss-alarmmgr-5dd8bb8b4f-h6rtb                    1/1     Running   0             2m52s
      prod-fss-auth-6875676df7-9b74r                        1/1     Running   0             3m42s
      prod-fss-catalog-7c5478d8ff-t546z                     1/1     Running   0             3m36s
      prod-fss-cfggen-6f5958bdf4-m6s54                      1/1     Running   0             3m42s
      prod-fss-cfgsync-6889dfc69d-t99mg                     1/1     Running   0             3m42s
      prod-fss-connect-69d8d49799-96w7k                     1/1     Running   0             3m42s
      prod-fss-da-bf8f8d456-6pmlr                           1/1     Running   0             3m42s
      prod-fss-da-bf8f8d456-h9z6j                           1/1     Running   0             3m42s
      prod-fss-da-bf8f8d456-l9fjh                           1/1     Running   0             3m26s
      prod-fss-da-bf8f8d456-lqzmn                           1/1     Running   0             3m41s
      prod-fss-da-bf8f8d456-nqbnz                           1/1     Running   0             75s
      prod-fss-da-bf8f8d456-ptd7t                           1/1     Running   0             3m41s
      prod-fss-da-bf8f8d456-tghht                           1/1     Running   0             3m31s
      prod-fss-da-bf8f8d456-xn5s5                           1/1     Running   0             83s
      prod-fss-deviationmgr-acl-6c88f46886-pl4bm            1/1     Running   0             3m42s
      prod-fss-deviationmgr-bfd-79bff7cd9d-5qst2            1/1     Running   0             3m42s
      prod-fss-deviationmgr-interface-77574648bf-zhrwk      1/1     Running   0             3m41s
      prod-fss-deviationmgr-netinst-79d9484657-jsp67        1/1     Running   0             3m41s
      prod-fss-deviationmgr-platform-868fc5fdc7-z7xzk       1/1     Running   0             3m41s
      prod-fss-deviationmgr-qos-6ffcfcbff5-2f2mj            1/1     Running   0             3m41s
      prod-fss-deviationmgr-routingpolicy-c9c5d56db-5zm6r   1/1     Running   0             3m40s
      prod-fss-deviationmgr-system-6cd48bdb67-42lb8         1/1     Running   0             3m40s
      prod-fss-dhcp-c595df9fc-lx4ln                         1/1     Running   0             16m
      prod-fss-filemgr-7546d4945b-xhs2h                     1/1     Running   0             3m40s
      prod-fss-imagemgr-5bbddc7995-fj5vm                    1/1     Running   0             3m28s
      prod-fss-intentmgr-5c55d858d9-62cc6                   1/1     Running   0             2m55s
      prod-fss-inventory-7d5544bf6f-d6cm5                   1/1     Running   0             3m30s
      prod-fss-labelmgr-bdd847459-b5qkm                     1/1     Running   0             3m39s
      prod-fss-maintmgr-d95457cdf-45vtx                     1/1     Running   0             2m39s
      prod-fss-mgmtstack-7f67b85c97-hj6rd                   1/1     Running   0             3m39s
      prod-fss-oper-da-7cc49477f-ds2hk                      1/1     Running   0             80s
      prod-fss-oper-da-7cc49477f-fglt4                      1/1     Running   0             3m38s
      prod-fss-oper-da-7cc49477f-lmghk                      1/1     Running   0             3m39s
      prod-fss-oper-da-7cc49477f-mp8cz                      1/1     Running   0             61s
      prod-fss-oper-da-7cc49477f-pmkkc                      1/1     Running   0             3m37s
      prod-fss-oper-da-7cc49477f-rl2f8                      1/1     Running   0             72s
      prod-fss-oper-da-7cc49477f-vffjz                      1/1     Running   0             72s
      prod-fss-oper-da-7cc49477f-wprbk                      1/1     Running   0             3m37s
      prod-fss-oper-topomgr-665bdb7bcc-hwgnm                1/1     Running   0             3m12s
      prod-fss-topomgr-7756dbc5dc-pnfhp                     1/1     Running   0             3m38s
      prod-fss-transaction-66fc8f4d75-bjxdp                 1/1     Running   0             2m54s
      prod-fss-version-5bc8c875f6-5cpb8                     1/1     Running   0             2m53s
      prod-fss-web-7967d75595-x7dpb                         1/1     Running   0             3m38s
      prod-fss-workloadmgr-77f49c9dbb-gbz74                 1/1     Running   0             3m38s
      prod-fss-ztp-644d8c689d-cdn5s                         1/1     Running   0             3m38s
      prod-keycloak-0                                       1/1     Running   0             16m
      prod-mongodb-arbiter-0                                1/1     Running   0             16m
      prod-mongodb-primary-0                                1/1     Running   0             16m
      prod-mongodb-secondary-0                              1/1     Running   0             16m
      prod-neo4j-core-0                                     1/1     Running   0             16m
      prod-postgresql-0                                     1/1     Running   0             16m
      [root@fss-deployer ~]#
      
      
      
  5. The upgrade procedure replaces the software catalog. Changes that you made to the software catalog will be lost. If you have added new SRL images in the software catalog, you need to modify the software catalog and then upload the SRL images.
    For instructions, see Updating software catalog to include new versions in the Fabric Services System User Guide. You need to complete this step before you access the Fabric Services System for any changes.