LAG support

For the OpenShift and Kubernetes plugin, two types of bonds are supported for LAGs:

Mode-1 bond, Active-Backup. This bond:
- requires autonomous ports
- does not require a LAG in the fabric
- creates sub-interfaces for individual ports in the fabric
Mode-4 bond, 802.3ad LACP (Active/Active). This bond:
- requires an enabled, LACP-negotiated Ethernet channel
- creates a LAG sub-interface

For MH-LAGs, connectivity is supported from the FSC node to multiple SR Linux leaf nodes.

Bond configuration on the FSC cluster compute nodes is performed following the OpenShift recommended method of using the NMState operator.

Sample configurations

The following is an example of a mode-4 bond with four member ports:

apiVersion: nmstate.io/v1
kind: NodeNetworkConfigurationPolicy
metadata:
  name: bond0-worker1
spec:
  nodeSelector:
    kubernetes.io/hostname: blrfsctb01-worker1.lab.fsc.fss.nokia.com
  desiredState:
    interfaces:
    - name: bond0
      description: Bond0 config on worker1
      type: bond
      state: up
      link-aggregation:
        mode: 802.3ad
        options:
          miimon: '140'
        port:
          - enp1s0f0
          - enp1s0f1
          - enp1s0f2
          - enp1s0f3
      mtu: 1450

The following is an example of a mode 1 bond with four member ports:

apiVersion: nmstate.io/v1
kind: NodeNetworkConfigurationPolicy
metadata:
  name: bond0-worker1
spec:
  nodeSelector:
    kubernetes.io/hostname: blrfsctb01-worker1.lab.fsc.fss.nokia.com
  desiredState:
    interfaces:
    - name: bond0
      description: Bond0 config on worker1
      type: bond
      state: up
      link-aggregation:
        mode: active-backup
        options:
          miimon: '140'
        port:
          - enp1s0f0
          - enp1s0f1
          - enp1s0f2
          - enp1s0f3
      mtu: 1450

The following is an example of a NAD pointing to Bond0:

NAD1 (Mode4 bond):

NAD1(Mode4 bond)-
apiVersion: "k8s.cni.cncf.io/v1"
kind: NetworkAttachmentDefinition
metadata:
  name: nad1
  namespace: fsc1
spec:
  config: '{
      "cniVersion": "0.4.0",
      "name": "nad1",
      "plugins": [
        {
          "type": " macvlan",
          "master": "bond0",
          "mode": "bridge",
          "ipam": {
            "type": "whereabouts",
            "range": "13.1.1.1/24",
            "gateway": "13.1.1.254"
          }
        }
      ]
    }'

NAD2(Mode1 bond):


apiVersion: "k8s.cni.cncf.io/v1"
kind: NetworkAttachmentDefinition
metadata:
  name: nad2
  namespace: fsc1
spec:
  config: '{
      "cniVersion": "0.4.0",
      "name": "nad2",
      "plugins": [
        {
          "type": " macvlan",
          "master": "bond0",
          "mode": "bridge",
          "ipam": {
            "type": "whereabouts",
            "range": "13.1.1.1/24",
            "gateway": "13.1.1.254"
          }
        }
      ]
    }'

With Pod1 pointing to NAD1 and Pod2 pointing to NAD2, the following is an example of the Workload Interface related to the Mode4 bond and the Mode1 bond.

Workload Interface output for Mode4 Bond:


- apiVersion: fsc.fss.nokia.com/v1
  kind: WorkloadInterface
  metadata:
    annotations:
      fsc/metadata: '{"uid":"74d1134b-1536-4713-b395-faeb68a84dcf","creationTimestamp":"2023-09-08T06:23:15Z"}'
    creationTimestamp: "2023-09-08T06:23:15Z"
    generation: 1
    name: blrfsctb01-worker2.lab.fsc.fss.nokia.com-bond0-fsc1-nad1
    namespace: fsc-system
    resourceVersion: "29792311"
    uid: 6f371eb9-fefe-456b-841e-125880509a43
  spec:
    cni: fsc1/nad1
    serverInterface:
      interface: bond0
      isLag: true
      lagMode: LAGMODE4_8023AD
      member:
      - member-interface: enp1s0f0
      - member-interface: enp1s0f1
      - member-interface: enp1s0f2
      - member-interface: enp1s0f3
      node: blrfsctb01-worker2.lab.fsc.fss.nokia.com
      vlanEnd: "0"
      vlanStart: "0"
      vlanType: untagged
  status:
    connectStatus: Sync-Done
    intfSyncStatus:
      fsc1/nad1: Sync-Done
    pendingPodkeys: {}
    syncedPodkeys:
      blrfsctb01-worker2.lab.fsc.fss.nokia.com:4acf7cebfc637638e54c280a36d7b2dc1c2e72cf91bf9f0d36bc4f984e722c61-net1: fsc1/pod1-64f74c7b6c-kv8ss

Workload Interface output for Mode1 Bond:


- apiVersion: fsc.fss.nokia.com/v1
  kind: WorkloadInterface
  metadata:
    annotations:
      fsc/metadata: '{"uid":"fc2ea1b2-e988-4309-812a-268593baa006","creationTimestamp":"2023-09-08T06:23:01Z"}'
    creationTimestamp: "2023-09-08T06:23:01Z"
    generation: 1
    name: blrfsctb01-worker1.lab.fsc.fss.nokia.com-bond0-fsc1-nad2
    namespace: fsc-system
    resourceVersion: "29791307"
    uid: f22d1313-01ed-481c-b182-19ffecd1197a
  spec:
    cni: fsc1/nad2
    serverInterface:
      interface: bond0
      isLag: true
      lagMode: LAGMODE1_ACTIVEBACKUP
      member:
      - member-interface: enp1s0f0
      - member-interface: enp1s0f1
      - member-interface: enp1s0f2
      - member-interface: enp1s0f3
      node: blrfsctb01-worker1.lab.fsc.fss.nokia.com
      vlanEnd: "0"
      vlanStart: "0"
      vlanType: untagged
  status:
    connectStatus: Sync-Done
    intfSyncStatus:
      fsc1/nad2: Sync-Done
    pendingPodkeys: {}
    syncedPodkeys:
      blrfsctb01-worker1.lab.fsc.fss.nokia.com:6bf126d6bfa2760996407a73ad2cc9178a06e974e693ec51ae8447b565b099a7-net1: fsc1/pod2-8669b46ddc-g6q6n

Note:

On a Fabric Services Controller application (FSC) deployed cluster worker node, some LAG events are not supported. These include:
- the dynamic removal or addition of LAG members
- the deletion of a LAG or changing LAG mode while user deployments are present on that LAG
To make changes to a LAG on an FSC deployed cluster worker node, you must delete all existing user deployments present on that LAG, make the required changes to the LAG configuration, and then re-deploy.
On an FSC deployed cluster worker node, Bond/LAG feature support is available only for the "ipvlan" and "macvlan" cni types. For the "sriov" cni type, Bond support is not available.
On an FSC deployed cluster worker node, a maximum of four LAG members per bond is supported.