Troubleshooting

Logs

By default the following logs are available:

  • FSC Controller/Manager log file on the master nodes: /var/log/fsc-data/controller/logs/fsc-controller-manager.log
  • FSC Helper CNI log file on all nodes: /var/log/fsc-data/cni/fsc-cni.log

FSC Global resource

A new custom resource is defined called FSC Global (fscGlobal) which contains information about the state of the Fabric Services System OpenShift integration.

It includes only a set of status fields, as shown below:

[fsc-helper@blrfsctb02-fsc-helper ~]$ oc get fscGlobal fscglobal -n fsc-system
NAME        AGE
fscglobal   69s
[fsc-helper@blrfsctb02-fsc-helper ~]$ oc describe fscglobal fscglobal -n fsc-system
Name:         fscglobal
Namespace:    fsc-system
Labels:       <none>
Annotations:  <none>
API Version:  fsc.fss.nokia.com/v1
Kind:         FscGlobal
Metadata:
  Creation Timestamp:  2023-05-22T09:49:55Z
  Generation:          1
  Resource Version:    36895377
  UID:                 91f5b160-d76c-4fcc-8e71-c06d91ad750b
Status:
  Controller Audit Status:    Done
  Controller Connect Status:  StateClientRunning
  Deployment Admin State:     AdminUp
  Deployment Id:              461741525005517023
  Deployment Name:            k8s-deployment-name
  Fsc Cni Injection Mode:     automatic
  Fsc Controller Replicas:    1
  Plugin Id:                  k8s-plugin-id
  Pod Name:                   prod-fss-fsc-controller-manager-86c48fb667-cp2mm
Events:                       <none>
[fsc-helper@blrfsctb02-fsc-helper ~]$

The ControllerAuditStatus highlights the latest state of the Audit mechanism of the FSC Controller, which can be any of the following:

  • Pending
  • InProgress
  • Done

The ControllerConnectStatus represents the status of the FSC Controller connectivity to the Fabric Services System Connect service. Possible values are:

  • StateClientInit
  • StateClientRestart
  • StateClientRegister
  • StateClientAudit
  • StateClientDisconnect
  • StateClientRunning

Action required before removing the OpenShift Plugin

The FSC installation creates the fsc-system namespace in the Kubernetes environment, and that namespace is used by FSC application.

While uninstalling the FSC application, the fsc-system namespace is also deleted as part of deletion process of the FSC application. For this reason, you must delete all artifacts that are part of the fsc-system namespace before removing the plugin.

Deleting these artifacts prevents the fsc-system namespace from becoming stuck in the Terminating state while it awaits the deletion of the resources that are using this namespace, and the FSC Helm delete process being unable to delete all FSC resources cleanly due to the namespace remaining stuck in Terminating state.