Performing a software upgrade

You use the fss-upgrade.sh script to perform the software upgrade steps. The command syntax is as follows:
[root@fss-deployer ~]# /root/bin/fss-upgrade.sh
Usage: /root/bin/fss-upgrade.sh configure <config-file> - Reads config file and configures accordingly.
 ./bin/fss-upgrade.sh discover - Discovers cluster.
 ./bin/fss-upgrade.sh upgrade - Upgrades FSS application.
 ./bin/fss-upgrade.sh status - Prints status of cluster and FSS Chart information.
 ./bin/fss-upgrade.sh clean - Cleans temporary files.
 ./bin/fss-upgrade.sh [help | -h] - Prints usage.

This script updates the entire Fabric Services System application using the new Helm charts.

Ensure that you have completed the steps described in Preparing for software upgrade. You execute the steps in this procedure while logged in to the new deployer VM that you created in Preparing for software upgrade.
  1. Update the configuration.
    In the following example, the input configuration file is customer-input.json.
    [root@fss-deployer ~]# /root/bin/fss-upgrade.sh configure customer-input.json
        Timesync service is running on 192.168.59.195  Time difference is 0 seconds
        Timesync service is running on 192.168.59.129  Time difference is 0 seconds
        Timesync service is running on 192.168.59.127  Time difference is 0 seconds
        Timesync service is running on 192.168.59.201  Time difference is 0 seconds
        Timesync service is running on 192.168.59.183  Time difference is 0 seconds 
        Timesync service is running on 192.168.59.190  Time difference is 0 seconds
      Maximum time difference between nodes 0 seconds
      Successfully configured. Please run /root/bin/fss-upgrade.sh upgrade
    Note: Do not execute the fss-upgrade.sh upgrade command as indicated in the output. Continue with Step 2.
  2. Discover the cluster.
    [root@fss-deployer ~]# /root/bin/fss-upgrade.sh discover
    Trying to discover 192.168.59.195
    Trying to discover 192.168.59.129
    Trying to discover 192.168.59.127
    NAME                      STATUS   ROLES                  AGE   VERSION
    fss-compute-01   Ready    control-plane,master   21m   v1.23.1
    fss-compute-02   Ready    control-plane,master   21m   v1.23.1
    fss-compute-03   Ready    control-plane,master   21m   v1.23.1
    fss-compute-04   Ready    <none>                 20m   v1.23.1
    fss-compute-05   Ready    <none>                 20m   v1.23.1
    fss-compute-06   Ready    <none>                 20m   v1.23.1
    FSS release discovered fss-FSS_22_8_B1-charts-v22.8.2-6
  3. Upgrade the Fabric Services System application.
    [root@fss-deployer ~]# /root/bin/fss-upgrade.sh upgrade
    NAME             STATUS   ROLES                  AGE   VERSION
    fss-compute-01   Ready    control-plane,master   21m   v1.23.1
    fss-compute-02   Ready    control-plane,master   21m   v1.23.1
    fss-compute-03   Ready    control-plane,master   21m   v1.23.1
    fss-compute-04   Ready    <none>                 20m   v1.23.1
    fss-compute-05   Ready    <none>                 20m   v1.23.1
    fss-compute-06   Ready    <none>                 20m   v1.23.1
    FSS will be upgraded from fss-FSS_22_8_B1-charts-v22.8.2-6 to fss-FSS_22_12_B1-charts v22.12.1-14 : Are you sure [YyNn]? y
    Upgrade in progress...
    W1208 05:20:03.982626    5196 warnings.go:67] policy/v1beta1 PodDisruptionBudget is deprecated in v1.21+, unavailable in v1.25+; use policy/v1 PodDisruptionBudget
    W1208 05:20:03.991013    5196 warnings.go:67] policy/v1beta1 PodDisruptionBudget is deprecated in v1.21+, unavailable in v1.25+; use policy/v1 PodDisruptionBudget
    W1208 05:20:04.000377    5196 warnings.go:67] policy/v1beta1 PodDisruptionBudget is deprecated in v1.21+, unavailable in v1.25+; use policy/v1 PodDisruptionBudget
    W1208 05:20:04.007284    5196 warnings.go:67] policy/v1beta1 PodDisruptionBudget is deprecated in v1.21+, unavailable in v1.25+; use policy/v1 PodDisruptionBudget
    W1208 05:20:04.015925    5196 warnings.go:67] policy/v1beta1 PodDisruptionBudget is deprecated in v1.21+, unavailable in v1.25+; use policy/v1 PodDisruptionBudget
    W1208 05:20:04.041830    5196 warnings.go:67] policy/v1beta1 PodDisruptionBudget is deprecated in v1.21+, unavailable in v1.25+; use policy/v1 PodDisruptionBudget
    W1208 05:20:04.048555    5196 warnings.go:67] policy/v1beta1 PodDisruptionBudget is deprecated in v1.21+, unavailable in v1.25+; use policy/v1 PodDisruptionBudget
    W1208 05:20:04.055468    5196 warnings.go:67] policy/v1beta1 PodDisruptionBudget is deprecated in v1.21+, unavailable in v1.25+; use policy/v1 PodDisruptionBudget
    W1208 05:20:04.070000    5196 warnings.go:67] policy/v1beta1 PodDisruptionBudget is deprecated in v1.21+, unavailable in v1.25+; use policy/v1 PodDisruptionBudget
    Release "prod" has been upgraded. Happy Helming!
    NAME: prod
    LAST DEPLOYED: Thu Dec  8 05:20:00 2022
    NAMESPACE: default
    STATUS: deployed
    REVISION: 2
    NOTES:
       Checking for FSS pods
           Waiting for  default  cp-zookeeper   3   Running
          Waiting for FSS pods to comeup
           Waiting for  default  cp-zookeeper   3   Running
          Waiting for FSS pods to comeup
           Waiting for  default  cp-zookeeper   3   Running
          Waiting for FSS pods to comeup
           Waiting for  default  fss-auth   1   Running
          Waiting for FSS pods to comeup
           Waiting for  default  fss-auth   1   Running
          Waiting for FSS pods to comeup
       All FSS pods are running
       Checking for FSS digitalsandbox pods
       FSS digital sandbox pods are running
       Checking for digitalsandbox pods
       Digital sandbox pods are running
     
       FSS is ready, you can access FSS using https://myhost.mydomain.com
    
  4. Verify that the upgrade is successful.
    1. Check the status of the compute nodes.
      The STATUS field should show Ready.
      [root@fss-deployer ~]# ./bin/fss-upgrade.sh status
      NAME             STATUS   ROLES                  AGE   VERSION
      fss-compute-01   Ready    control-plane,master   27m   v1.23.1
      fss-compute-02   Ready    control-plane,master   27m   v1.23.1
      fss-compute-03   Ready    control-plane,master   27m   v1.23.1
      fss-compute-04   Ready    <none>                 26m   v1.23.1
      fss-compute-05   Ready    <none>                 26m   v1.23.1 
      fss-compute-06   Ready    <none>                 26m   v1.23.1 
      FSS release discovered: fss-FSS_22_12_B1-charts-v22.12.1-14 ; Deployer packages FSS release:  fss-FSS_22_12_B1-charts v22.12.1-14
         Checking for FSS pods
         All FSS pods are running
         Checking for FSS digitalsandbox pods
         FSS digital sandbox pods are running
         Checking for digitalsandbox pods
         Digital sandbox pods are running
       
         FSS is ready, you can access FSS using https://myhost.mydomain.com
      
    2. Display the Helm chart version.
      [root@fss-deployer ~]# export KUBECONFIG=/var/lib/fss/config.fss
      [root@fss-deployer ~]# helm list
      NAME    NAMESPACE       REVISION        UPDATED                                 STATUS          CHART                                   APP VERSION
      prod    default         2               2022-10-27 18:06:07.835241335 +0000 UTC deployed        fss-FSS_22_12_B1-charts-v22.12.1-14        1.0
      
    3. Check the status of system services and processes.
      The upgrade is in progress until the STATUS field for all pods shows Running.
      [root@fss-deployer ~]# kubectl get pods
      NAME                                                   READY   STATUS        RESTARTS      AGE
      prod-cp-kafka-0                                        2/2     Running       2 (11m ago)   13m
      prod-cp-kafka-1                                        2/2     Running       1 (11m ago)   12m
      prod-cp-kafka-2                                        2/2     Running       0             12m
      prod-cp-zookeeper-0                                    2/2     Running       0             13m
      prod-cp-zookeeper-1                                    2/2     Running       0             11m
      prod-cp-zookeeper-2                                    2/2     Running       0             11m
      prod-fss-alarmmgr-6678f4d49c-xz77v                     1/1     Terminating   0             13m
      prod-fss-auth-6875676df7-9b74r                         1/1     Running       0             33s
      prod-fss-auth-79ff5f495c-s4t7m                         0/1     Terminating   0             13m
      prod-fss-catalog-7c5478d8ff-t546z                      0/1     Init:0/1      0             27s
      prod-fss-cfggen-5d8f94f74b-57nd7                       1/1     Running       0             13m
      prod-fss-cfggen-6f5958bdf4-m6s54                       0/1     Init:0/1      0             33s
      prod-fss-cfgsync-5b69b7f854-l9lx5                      1/1     Running       0             13m
      prod-fss-cfgsync-6889dfc69d-t99mg                      0/1     Init:0/1      0             33s
      prod-fss-connect-69d8d49799-96w7k                      0/1     Init:0/1      0             33s
      prod-fss-connect-75df8ff7f9-vwfgj                      1/1     Running       0             13m
      prod-fss-da-69f8676b6d-4kjsn                           0/1     Terminating   0             13m
      prod-fss-da-69f8676b6d-cp5v6                           1/1     Terminating   0             13m
      prod-fss-da-69f8676b6d-fj7db                           1/1     Running       0             13m
      prod-fss-da-69f8676b6d-mnz9b                           1/1     Running       0             13m
      prod-fss-da-69f8676b6d-ph42r                           1/1     Running       0             13m
      prod-fss-da-69f8676b6d-vgflj                           0/1     Terminating   0             13m
      prod-fss-da-69f8676b6d-zg2pb                           1/1     Running       0             13m
      prod-fss-da-69f8676b6d-zt7z6                           1/1     Terminating   0             13m
      prod-fss-da-bf8f8d456-6pmlr                            0/1     Init:0/1      0             33s
      prod-fss-da-bf8f8d456-h9z6j                            1/1     Running       0             33s
      prod-fss-da-bf8f8d456-l9fjh                            0/1     Init:0/1      0             17s
      prod-fss-da-bf8f8d456-lqzmn                            1/1     Running       0             32s
      prod-fss-da-bf8f8d456-ptd7t                            0/1     Init:0/1      0             32s
      prod-fss-da-bf8f8d456-tghht                            0/1     Init:0/1      0             22s
      prod-fss-deviationmgr-acl-6c88f46886-pl4bm             0/1     Init:0/1      0             33s
      prod-fss-deviationmgr-acl-d8949f794-2dl79              1/1     Running       0             13m
      prod-fss-deviationmgr-bfd-68f5d7988b-q8vks             1/1     Running       0             13m
      prod-fss-deviationmgr-bfd-79bff7cd9d-5qst2             0/1     Init:0/1      0             33s
      prod-fss-deviationmgr-interface-769c5d46f-l9wrc        1/1     Running       0             13m
      prod-fss-deviationmgr-interface-77574648bf-zhrwk       0/1     Init:0/1      0             32s
      prod-fss-deviationmgr-netinst-6f455d75d5-gqftv         1/1     Running       0             13m
      prod-fss-deviationmgr-netinst-79d9484657-jsp67         0/1     Init:0/1      0             32s
      prod-fss-deviationmgr-platform-589dd98c4f-l67dp        1/1     Running       0             13m
      prod-fss-deviationmgr-platform-868fc5fdc7-z7xzk        0/1     Init:0/1      0             32s
      prod-fss-deviationmgr-qos-567cbfc7ff-twblt             1/1     Running       0             13m
      prod-fss-deviationmgr-qos-6ffcfcbff5-2f2mj             0/1     Init:0/1      0             32s
      prod-fss-deviationmgr-routingpolicy-7cbcfc858b-rxf7p   1/1     Running       0             13m
      prod-fss-deviationmgr-routingpolicy-c9c5d56db-5zm6r    0/1     Init:0/1      0             31s
      prod-fss-deviationmgr-system-6549bc48ff-2qttp          1/1     Running       0             13m
      prod-fss-deviationmgr-system-6cd48bdb67-42lb8          0/1     Init:0/1      0             31s
      prod-fss-dhcp-c595df9fc-lx4ln                          1/1     Running       0             13m
      prod-fss-filemgr-687fc88b4d-hc8bz                      1/1     Running       0             13m
      prod-fss-filemgr-7546d4945b-xhs2h                      0/1     Init:0/1      0             31s
      prod-fss-imagemgr-5bbddc7995-fj5vm                     0/1     Init:0/1      0             19s
      prod-fss-intentmgr-78fcdf84df-fzmzm                    0/1     Terminating   0             13m
      prod-fss-inventory-7d5544bf6f-d6cm5                    0/1     Init:0/1      0             21s
      prod-fss-labelmgr-7db67c68cf-482kw                     1/1     Running       0             13m
      prod-fss-labelmgr-bdd847459-b5qkm                      0/1     Init:0/1      0             30s
      prod-fss-maintmgr-76d755c8cc-m6njl                     1/1     Terminating   0             13m
      prod-fss-mgmtstack-68b8fc9fdf-6xt7f                    1/1     Running       0             13m
      prod-fss-mgmtstack-7f67b85c97-hj6rd                    0/1     Init:0/1      0             30s
      prod-fss-oper-da-6556b58fc5-29tw6                      1/1     Running       0             13m
      prod-fss-oper-da-6556b58fc5-789nx                      0/1     Terminating   0             13m
      prod-fss-oper-da-6556b58fc5-bpnvg                      1/1     Running       0             13m
      prod-fss-oper-da-6556b58fc5-bvvkc                      1/1     Running       0             13m
      prod-fss-oper-da-6556b58fc5-k7xqz                      1/1     Running       0             13m
      prod-fss-oper-da-6556b58fc5-rlqlv                      0/1     Terminating   0             13m
      prod-fss-oper-da-6556b58fc5-wn2bb                      1/1     Running       0             13m
      prod-fss-oper-da-6556b58fc5-zvkx2                      1/1     Running       0             13m
      prod-fss-oper-da-7cc49477f-fglt4                       0/1     Init:0/1      0             29s
      prod-fss-oper-da-7cc49477f-lmghk                       0/1     Init:0/1      0             30s
      prod-fss-oper-da-7cc49477f-pmkkc                       0/1     Init:0/1      0             28s
      prod-fss-oper-da-7cc49477f-wprbk                       0/1     Init:0/1      0             28s
      prod-fss-oper-topomgr-665bdb7bcc-hwgnm                 0/1     Pending       0             3s
      prod-fss-topomgr-7756dbc5dc-pnfhp                      0/1     Init:0/1      0             29s
      prod-fss-topomgr-77cd566695-5w5mx                      1/1     Running       0             13m
      prod-fss-transaction-5756574f99-clfv5                  1/1     Terminating   0             13m
      prod-fss-version-b565f896-kcktt                        0/1     Terminating   0             13m
      prod-fss-web-5d5bbf4c67-gd6bf                          1/1     Running       0             13m
      prod-fss-web-7967d75595-x7dpb                          0/1     Init:0/1      0             29s
      prod-fss-workloadmgr-77f49c9dbb-gbz74                  0/1     Init:0/1      0             29s
      prod-fss-workloadmgr-c99db6bb6-fft5q                   1/1     Running       0             13m
      prod-fss-ztp-59675c6ffd-pzpxj                          1/1     Running       0             13m
      prod-fss-ztp-644d8c689d-cdn5s                          0/1     Init:0/1      0             29s
      prod-keycloak-0                                        1/1     Running       0             13m
      prod-mongodb-arbiter-0                                 1/1     Running       0             13m
      prod-mongodb-primary-0                                 1/1     Running       0             13m
      prod-mongodb-secondary-0                               1/1     Running       0             13m
      prod-neo4j-core-0                                      1/1     Running       0             13m
      prod-postgresql-0                                      1/1     Running       0             13m
      [root@fss-deployer ~]#
      
      [root@fss-deployer ~]# kubectl get pods
      NAME                                                  READY   STATUS    RESTARTS      AGE
      prod-cp-kafka-0                                       2/2     Running   2 (14m ago)   16m
      prod-cp-kafka-1                                       2/2     Running   1 (14m ago)   15m
      prod-cp-kafka-2                                       2/2     Running   0             15m
      prod-cp-zookeeper-0                                   2/2     Running   0             16m
      prod-cp-zookeeper-1                                   2/2     Running   0             14m
      prod-cp-zookeeper-2                                   2/2     Running   0             14m
      prod-fss-alarmmgr-5dd8bb8b4f-h6rtb                    1/1     Running   0             2m52s
      prod-fss-auth-6875676df7-9b74r                        1/1     Running   0             3m42s
      prod-fss-catalog-7c5478d8ff-t546z                     1/1     Running   0             3m36s
      prod-fss-cfggen-6f5958bdf4-m6s54                      1/1     Running   0             3m42s
      prod-fss-cfgsync-6889dfc69d-t99mg                     1/1     Running   0             3m42s
      prod-fss-connect-69d8d49799-96w7k                     1/1     Running   0             3m42s
      prod-fss-da-bf8f8d456-6pmlr                           1/1     Running   0             3m42s
      prod-fss-da-bf8f8d456-h9z6j                           1/1     Running   0             3m42s
      prod-fss-da-bf8f8d456-l9fjh                           1/1     Running   0             3m26s
      prod-fss-da-bf8f8d456-lqzmn                           1/1     Running   0             3m41s
      prod-fss-da-bf8f8d456-nqbnz                           1/1     Running   0             75s
      prod-fss-da-bf8f8d456-ptd7t                           1/1     Running   0             3m41s
      prod-fss-da-bf8f8d456-tghht                           1/1     Running   0             3m31s
      prod-fss-da-bf8f8d456-xn5s5                           1/1     Running   0             83s
      prod-fss-deviationmgr-acl-6c88f46886-pl4bm            1/1     Running   0             3m42s
      prod-fss-deviationmgr-bfd-79bff7cd9d-5qst2            1/1     Running   0             3m42s
      prod-fss-deviationmgr-interface-77574648bf-zhrwk      1/1     Running   0             3m41s
      prod-fss-deviationmgr-netinst-79d9484657-jsp67        1/1     Running   0             3m41s
      prod-fss-deviationmgr-platform-868fc5fdc7-z7xzk       1/1     Running   0             3m41s
      prod-fss-deviationmgr-qos-6ffcfcbff5-2f2mj            1/1     Running   0             3m41s
      prod-fss-deviationmgr-routingpolicy-c9c5d56db-5zm6r   1/1     Running   0             3m40s
      prod-fss-deviationmgr-system-6cd48bdb67-42lb8         1/1     Running   0             3m40s
      prod-fss-dhcp-c595df9fc-lx4ln                         1/1     Running   0             16m
      prod-fss-filemgr-7546d4945b-xhs2h                     1/1     Running   0             3m40s
      prod-fss-imagemgr-5bbddc7995-fj5vm                    1/1     Running   0             3m28s
      prod-fss-intentmgr-5c55d858d9-62cc6                   1/1     Running   0             2m55s
      prod-fss-inventory-7d5544bf6f-d6cm5                   1/1     Running   0             3m30s
      prod-fss-labelmgr-bdd847459-b5qkm                     1/1     Running   0             3m39s
      prod-fss-maintmgr-d95457cdf-45vtx                     1/1     Running   0             2m39s
      prod-fss-mgmtstack-7f67b85c97-hj6rd                   1/1     Running   0             3m39s
      prod-fss-oper-da-7cc49477f-ds2hk                      1/1     Running   0             80s
      prod-fss-oper-da-7cc49477f-fglt4                      1/1     Running   0             3m38s
      prod-fss-oper-da-7cc49477f-lmghk                      1/1     Running   0             3m39s
      prod-fss-oper-da-7cc49477f-mp8cz                      1/1     Running   0             61s
      prod-fss-oper-da-7cc49477f-pmkkc                      1/1     Running   0             3m37s
      prod-fss-oper-da-7cc49477f-rl2f8                      1/1     Running   0             72s
      prod-fss-oper-da-7cc49477f-vffjz                      1/1     Running   0             72s
      prod-fss-oper-da-7cc49477f-wprbk                      1/1     Running   0             3m37s
      prod-fss-oper-topomgr-665bdb7bcc-hwgnm                1/1     Running   0             3m12s
      prod-fss-topomgr-7756dbc5dc-pnfhp                     1/1     Running   0             3m38s
      prod-fss-transaction-66fc8f4d75-bjxdp                 1/1     Running   0             2m54s
      prod-fss-version-5bc8c875f6-5cpb8                     1/1     Running   0             2m53s
      prod-fss-web-7967d75595-x7dpb                         1/1     Running   0             3m38s
      prod-fss-workloadmgr-77f49c9dbb-gbz74                 1/1     Running   0             3m38s
      prod-fss-ztp-644d8c689d-cdn5s                         1/1     Running   0             3m38s
      prod-keycloak-0                                       1/1     Running   0             16m
      prod-mongodb-arbiter-0                                1/1     Running   0             16m
      prod-mongodb-primary-0                                1/1     Running   0             16m
      prod-mongodb-secondary-0                              1/1     Running   0             16m
      prod-neo4j-core-0                                     1/1     Running   0             16m
      prod-postgresql-0                                     1/1     Running   0             16m
      [root@fss-deployer ~]#
      
      
      
  5. Restore previously added software catalog images.
    The upgrade procedure replaces the software catalog. Changes that you made to the software catalog are lost after the upgrade. If you previously added new SR Linus images to the software catalog, you need to modify the software catalog and then upload the SR Linux images. For instructions, see Adding a new software catalog in the Fabric Services System User Guide. You need to complete this step before you can access the Fabric Services System to make any change.