Usage

This section lists commands available for managing the OpenShift plugin.

Helper CNI injection

If the injectCni is set to true (the default value), FSC automatically injects the Helper CNI configuration in any Network Attachment Definition that is created in the platform. This assures the correct functionality of the Fabric Services System integration.

If the injection was disabled during the installation, add the following CNI configuration to the list of plugins for each Network Attachment Definition that requires fabric management:

{
  "type": "fsc-cni",
  "args": {
    "parent": "<nad-namespace>/<nad-name>",  
    "cnicache": "/var/lib/cni/fsc-cni"
  }
}

In this plugin definition, the <nad-namespace>/<nad-name> value must be changed to the actual namespace and name of the Network Attachment Definition. A complete example of an IPVLAN Network Attachment Definition looks like the following:

---
apiVersion: "k8s.cni.cncf.io/v1"
kind: NetworkAttachmentDefinition
metadata:
  name: def-nad9-port2
spec:
  config: '{
     "cniVersion": "0.3.1",
     "name": "def-nad9-port2",
     "plugins": [
       {
         "type": "ipvlan",
         "master": "fscintf2.2709",
         "mode": "l2",    
         "ipam": {
           "type": "whereabouts",
           "range": "29.1.1.1/24",
           "gateway": "29.1.1.254"
         }
       },
       {
         "type": "fsc-cni",
         "args": {
           "parent": "default/def-nad9-port2",  
           "cnicache": "/var/lib/cni/fsc-cni"
         }
       }
     ]
    }'

Note: When injectCni is set to true, FSC only injects the Helper CNI definition in the Network Attachment Definitions configured in the WorkloadConnectivityIntent (WCI) when the WCI is applied. Also, when the WCI is deleted, FSC removed the Helper CNI definition in the Network-Attachment-Definitions configured in the WCI. If a change is applied to a Network AttachmentdDefinition that is already present in the applied WCI, it is not added again; you must inject the Helper CNI in the updated Network AttachmentdDefinition.

Defining Workload Connectivity Intent resources

A Workload Connectivity Intent contains the network design for an application. Each Workload Connectivity Intent matches with a Workload VPN Intent inside the Fabric Services System.

Below is a n overview of the definition of a Workload Connectivity Intent.

---
apiVersion: fsc.fss.nokia.com/v1
kind: WorkloadConnectivityIntent
metadata:
  name: app01 # A name for this Workload Connectivity Intent
  namespace: fsc-system # Should always be the fsc-system namespace
spec:
  namespace: app01 # (Optional) The namespace to find the NADs
  type: "IRB" # Should always be IRB
  subnets: # List of Subnets (MAC VRFs) to create in the fabric
   - name: "frontend" # Name of the subnet 
     type: "bridged" # Should always be bridged
     cni: # List of NADs that need to connect into this subnet
       - "frontend-nad01" # A NAD name in the 'app01' namespace
       - "global-ns/frontend-shared" # A NAD in a different namespace
   - name: "backend" 
     type: "bridged"
     cni:
       - "backend-nad01" 
   - name: "database"
     type: "bridged"
      cni:
       - "db-nad01"

The above Workload Connectivity Intent results in a network design in the Fabric Services System that has a Workload VPN Intent named “app01” with three subnets: “frontend”, “backend” and “database”.

When a pod starts that refers to any of the referenced Network Attachment Definitions, the helper CNI and FSC ensure that the fabric is properly configured to provide connectivity for that pod on the specific worker node on which it is started.

Note: A Network Attachment Definition must exist before it can be referenced in a Workload Connectivity Intent, and it can only be referenced by one Workload Connectivity Intent.

Note: If you configure non-default namespace under the spec context, Network Attachment Definitions for that namespace only can be specified in the list of CNI under any subnet of that WorkloadConnectivityIntent.

Using the Fabric Services System Managed mode

The example Workload Connectivity Intent in Defining Workload Connectivity Intent resources is an example representing the use of the Cloud Managed mode (for more details, see Fabric Services System Connect workflows).

The OpenShift integration also supports the use of the Fabric Services System mode. In this scenario, the Workload Intents and Subnets are created in the Fabric Services System, without sub-interfaces, before the Workload Connectivity Intent gets created.

After the creation of the Workload Intents and Subnets in the Fabric Services System, the operator or administrator of OpenShift creates a Workload Connectivity Intent where the workload and each subnet refers to the pre-created resources by using the appropriate unique identifier (UUID).

An example of a Workload Connectivity Intent when using the Fabric Services System Managed mode:

apiVersion: fsc.fss.nokia.com/v1
kind: WorkloadConnectivityIntent
metadata:
  name: tenant1
  namespace: fsc-system
spec:
  type: "IRB"
  namespace: fsc-system
  fssWorkloadEvpnID: "433478043755872256" # The Workload Intent ID from FSS
  subnets:
    - name: "subnet1"
      type: "bridged"
      fssSubnetID: "433478096436330496" # The Subnet ID from FSS
      cni:
        - "nad1"

When using the Fabric Services System Manage mode, the following extra fields are mandatory in the Workload Connectivity Intent:

fssWorkloadEvpnID: The ID of the Workload Intent inside the Fabric Services System that was created. It must be configured at the root of the spec of the Workload Connectivity Intent.
fssSubnetID: The ID of a Subnet that is part of the Workload Intent referred to by the fssWorkloadEvpnID. This needs to be defined for each subnet in the Workload Connectivity Intent if a fssWorkloadEvpnID is configured.

Listing Workload Connectivity Intents

You can retrieve a list of existing Workload Connectivity Intents with the following command:

$ kubectl get workloadconnectivityintents.fsc.fss.nokia.com -n fsc-system
NAME    AGE
app01   17m
app02   17m

Inspecting a Workload Connectivity Intent

To retrieve all the details of a Workload Connectivity intent, run the following command:

$ kubectl describe workloadconnectivityintents.fsc.fss.nokia.com app01 -n fsc-system
Name:         app01
Namespace:    fsc-system
Labels:       <none>
Annotations:  <none>
API Version:  fsc.fss.nokia.com/v1
Kind:         WorkloadConnectivityIntent
Metadata:
  Creation Timestamp:  2022-07-28T06:09:19Z
  Finalizers:
    fsc.io/gWCFinalizer
  Generation:  2
  Managed Fields:
    API Version:  fsc.fss.nokia.com/v1
    Fields Type:  FieldsV1
    fieldsV1:
      f:metadata:
        f:finalizers:
          .:
          v:"fsc.io/gWCFinalizer":
      f:spec:
        f:managedid:
        f:subnets:
    Manager:      fsc-manager
    Operation:    Update
    Time:         2022-07-28T06:09:19Z
    API Version:  fsc.fss.nokia.com/v1
    Fields Type:  FieldsV1
    fieldsV1:
      f:metadata:
        f:annotations:
          .:
          f:kubectl.kubernetes.io/last-applied-configuration:
      f:spec:
        .:
        f:namespace:
        f:type:
    Manager:      kubectl-client-side-apply
    Operation:    Update
    Time:         2022-07-28T06:09:19Z
    API Version:  fsc.fss.nokia.com/v1
    Fields Type:  FieldsV1
    fieldsV1:
      f:status:
        .:
        f:connectstatus:
        f:crdstatus:
        f:subnetstatus:
    Manager:         fsc-manager
    Operation:       Update
    Subresource:     status
    Time:            2022-07-28T06:09:20Z
  Resource Version:  18598421
  UID:               cfdf73bb-ff85-45e2-aca0-0b1f3d7643e1
Spec:
  Managedid: 
  Namespace:  app01
  Subnets:
   - name: "frontend" # Name of the subnet 
     type: "bridged" # Should always be bridged
     cni: # List of NADs that need to connect into this subnet
       - "frontend-nad01" # A NAD name in the 'app01' namespace
       - "global-ns/frontend-shared" # A NAD in a different namespace
   - name: "backend" 
     type: "bridged"
     cni:
       - "backend-nad01" 
   - name: "database"
     type: "bridged"
      cni:
       - "db-nad01"
    Cni:
      frontend-nad01
      global-ns/frontend-shared
    Managedid: 
    Name:       frontend
    Type:       bridged
    Cni:
      backend-nad01
    Managedid: 
    Name:       backend
    Type:       bridged
    Cni:
      db-nad01
    Managedid: 
    Name:       database
    Type:       bridged
  Type:         IRB
Status:
  Connectstatus:  Sync-Done
  Crdstatus:      Queued-update-FSS
  Subnetstatus:
    Cnistatus:
      Connectedpods:  false
      Connectstatus:  Sync-Done
      Name:           frontend-nad01
      Namespace:      app01
      Connectedpods:  false
      Connectstatus:  Sync-Done
      Name:           global-ns/frontend-shared
      Namespace:      global-ns
    Connectstatus:    Sync-Done
    Name:             frontend
    Cnistatus:
      Connectedpods:  false
      Connectstatus:  Sync-Done
      Name:           backend-nad01
      Namespace:      app01
    Connectstatus:    Sync-Done
    Name:             backend
    Cnistatus:
      Connectedpods:  false
      Connectstatus:  Sync-Done
      Name:           db-nad01
      Namespace:      app01
    Connectstatus:    Sync-Done
    Name:             database
Events:               <none>

Potential Status fields for a Workload Connectivity Intent

The following status values can be returned in the different status fields of a Workload Connectivity Intent:

Connectedpods: Is “true” when there are Pods are running using this NAD. If there are no Pods using this NAD, the status is “false”.
Crdstatus: Can have the following values:
- "CNI-Validation-Failed" – Failed to validate the NAD presence.
- "Queued-update-FSS" – An update is queued to the Fabric Services System and the status is monitored by the Connect status.
- "Queued-delete-FSS" – A delete is queued to the Fabric Services System and the status is monitored by the Connect status.
Connectstatus: Can have the following values:
- Sync-Done indicates that a NAD is deployed in the Fabric Services System through the Connect service.
- Reg-Failed indicates the registration has failed.
- Sync-Pending indicates a request was sent to the Fabric Services System to create resources, but no response was received. A new attempt is made to achieve Sync-done at a regular interval.
- Sync-Deleted indicates a request to delete the resources was sent to the Fabric Services System and a successful response was received.
- Sync-Mark-Delete indicates a request to delete the resources was sent to the Fabric Services System, but no response was received.
- Sync-Add-Failed indicates a request to add or create the resources was sent to the Fabric Services System, and a failure response was received for some reason. No further attempts are made in this case.
- Sync-Del-Failed indicates a request to delete the resources was sent to the Fabric Services System and a failure response was received for some reason. No further attempts are made in this case.

Deleting a Workload Connectivity Intent

A Workload Connectivity Intent cannot be deleted if any pods are using the Network Attachment Definitions that are referenced in the Workload Connectivity Intent.

To delete a Workload Connectivity Intent, run the following command:

$ kubectl delete workloadconnectivityintents.fsc.fss.nokia.com app01 -n fsc-system

Working with Workload Interface resources

When a pod is started on a worker node, the Helper CNI creates Workload Interfaces to indicate which worker nodes, physical interfaces, and VLANs on those interfaces must be added as sub-interfaces for a specific Network Attachment Definition.

These Workload Interface resources should not be manipulated by the operator of the OpenShift cluster, and are under the full control of the Helper CNI and the FSC.

You can retrieve a list of Workload Interfaces with the following commands.

$ kubectl get workloadinterfaces.fsc.fss.nokia.com -n fsc-system
NAME                                      AGE
Worker1.lab.fsc.io-app01-frontend-nad01   17m
Worker2.lab.fsc.io-app01-frontend-nad01   17m

$ kubectl describe workloadinterfaces.fsc.fss.nokia.com worker1.lab.fsc.io-app01-frontend-nad01 -n fsc-system
Name:         worker1.lab.fsc.io-app01-frontend-nad01
Namespace:    fsc-system
Labels:       <none>
Annotations:  fsc/metadata: {"uid":"58542c1f-4a3c-4987-9140-8d3029a05e37","creationTimestamp":"2022-07-28T06:46:09Z"}
API Version:  fsc.fss.nokia.com/v1
Kind:         WorkloadInterface
Metadata:
  Creation Timestamp:  2022-07-28T06:46:09Z
  Generation:          1
  Managed Fields:
    API Version:  fsc.fss.nokia.com/v1
    Fields Type:  FieldsV1
    fieldsV1:
      f:spec:
        .:
        f:cni:
        f:server-interface:
          .:
          f:interface:
          f:node:
          f:vlan-end:
          f:vlan-start:
          f:vlan-type:
    Manager:      Go-http-client
    Operation:    Update
    Time:         2022-07-28T06:46:09Z
    API Version:  fsc.fss.nokia.com/v1
    Fields Type:  FieldsV1
    fieldsV1:
      f:status:
        .:
        f:connectstatus:
        f:pending-podkeys:
        f:synced-podkeys:
          .:
          f:worker1.lab.fsc.io:37721f66056e9e87038cf39a73e023b27046dabb7ec1b0a5d59dcfe6d53081a0-net1:
          f:worker1.lab.fsc.io:4df27e16cc53a660f3841a10522c02e254f722907c7be9f193965107750cca4f-net1:
          f:worker1.lab.fsc.io:c7acdb797dae90fa2318b47063493dbd24ca46c300ca7560cebdb0448c5dc74d-net1:
          f:worker1.lab.fsc.io:e21de0ba145d2576f8f1423b9d5dfece6fe399ec9db8996c05afbce837a36533-net1:
    Manager:         fsc-manager
    Operation:       Update
    Subresource:     status
    Time:            2022-07-28T06:46:09Z
  Resource Version:  18611818
  UID:               21fbed90-43fb-465f-bbfb-e5de7ee9fe9d
Spec:
  Cni:  app01/frontend-nad01
  Server - Interface:
    Interface:     enp6s0
    Node:          worker1.lab.fsc.io
    Vlan - End:    2006
    Vlan - Start:  2006
    Vlan - Type:   VLANTYPE_VALUE
Status:
  Connectstatus:  Sync-Done
  Pending - Podkeys:
  Synced - Podkeys:
    Worker1.lab.fsc.io:37721f66056e9e87038cf39a73e023b27046dabb7ec1b0a5d59dcfe6d53081a0-net1: blrtwo/blrtwo-def-nad10-dep-7bd5f9f9c4-ptmnx
    worker1.lab.fsc.io:4df27e16cc53a660f3841a10522c02e254f722907c7be9f193965107750cca4f-net1: blrtwo/blrtwo-def-nad10-dep-7bd5f9f9c4-lzjm7
    worker1.lab.fsc.io:c7acdb797dae90fa2318b47063493dbd24ca46c300ca7560cebdb0448c5dc74d-net1: blrtwo/blrtwo-def-nad10-dep-7bd5f9f9c4-dg6kt
    worker1.lab.fsc.io:e21de0ba145d2576f8f1423b9d5dfece6fe399ec9db8996c05afbce837a36533-net1: blrtwo/blrtwo-def-nad10-dep-7bd5f9f9c4-2xvnq
Events:     
<none>