Workflows and workflow definitions

A workflow is a sequence of steps required to perform some process.

The concept of a workflow is typically used in automation platforms to make an operational task reproducible; it is the logic or code required to execute the task. It is not too much of an extrapolation from the concept of a workflow to a CI/CD pipeline, which describes a sequence of actions to run, with workflow semantics between them.

In EDA, workflows can define the steps required to upgrade a TopoNode, validate connectivity in a VirtualNetwork, or perform a simple ping operation.

A workflow in EDA is implemented via an orchestrated Kubernetes Job. Workflows themselves are container images that take some input, perform some work, and provide some output.

Before running any workflows, the relevant Workflow Definitions must be loaded into the system.

Note: EDA 25.4 removes the Workflow resource, which was overly complex to RBAC. Instead, EDA now employs a typical CRD approach, in which CRDs within the applications manifest are tagged with the workflow boolean.

EDA supports interactions with workflows through the following means:

Through the API.
- Hitting any of the per-app workflow endpoints; for example, /workflows/interfaces.eda.nokia.com/v1alpha1/checkinterface, with a PUT or POST. These endpoints are synchronous - they only return OK or NOK when the corresponding workflow completes.
- Hitting the workflow endpoint; for example, /core/workflow/v1, with a PUT or POST.This endpoint is asynchronous - returning a workflow ID immediately, which can in turn be queried.
Through Kubernetes.
- Creating a resource that is marked as a workflow.
Through gRPC towards FlowEngine. - gRPC endpoint for workflow interactions. This is what APIServer uses when a workflow hits its REST API.

FlowEngine

In EDA workflows are supported through the FlowEngine - the controller behind the instantiation, status, and interaction with workflows. FlowEngine:

Listens for CRUD operations on WorkflowDefinitions. These define a workflow.
Listens for CRUD operations on CRDs that are tagged as workflows. These indicate a users desire to execute a given WorkflowDefinition.
On receiving a new workflow the FlowEngine:
- Validates the resources input against schema.
- Publishes a Kubernetes Job resource, referencing the container image provided in the corresponding WorkflowDefinition.
- Updates the status of the flow based on gRPC interactions.

Flow IDs are restarted at 1, and previously executed flows/currently running flows are lost in the event of a FlowEngine restart.

FlowEngine supports the following:

Loading workflows post-runtime.
- Apps are the packaging boundaries for workflows. For example, an owner of an app that deploys IP and MAC VRF services may also package a flow that tests those services post deployment.
- This means flows follow the same install/iterate mechanisms using a Manifest and the EDA Store.
Manual triggering of workflows.
Manual cancelling of workflows that are currently running.
Reporting the status of executed workflows.
General user interactions with workflows - the ability to block a flow and wait for user input.
Workflow hierarchy - one flow may be a parent of another.
Automatic creation of workflow IDs.
Interactions from workflows via gRPC and the EDK.
Interactions with users and other machines via edactl.
Writing of workflows in any language that supports gRPC and protobufs.
Non-blocking behavior; up to 256 workflows can be executing at the same time.
Note: To avoid excessively memory use by FlowEngine, EDA enforces the following:
- Only 256 parent workflows are persisted.
- New workflows push old workflows out - we tail drop history.
- There is a limit of 256 concurrently running workflows. New workflows are rejected if the system has reached this limit; actively running workflows are never dropped.
- This history includes stages and logs.
This history persists for the lifetime of FlowEngine. It does not persist and does not remain after a restart.

Workflow definitions

A WorkflowDefinition in EDA binds together two things:

a container image whose entrypoint executes a workflow
a name in the system

A workflow definition also allows its creator to define the input of the workflow, and its output.

A workflow definition can include the following fields:

The usual metadata, including:
- Name
- Labels
- Annotations
image: indicates a path to an image to use when executing this workflow.
jsonSchemaSpec: a full JSON schema defining the input the workflow accepts.
jsonSchemaStatus: a full JSON schema defining the output this workflow populates.

In EDA, both workflow and CI/CD functionality are supported through the WorkflowEngine; the controller behind the instantiation, status, and interaction with the Workflow and WorkflowDefinition resources.

Sample workflow definitions

A workflow is essentially a container image that is tied with some input - being the resource it is triggered against. The WorkflowDefinition resource contains this binding, for example:

apiVersion: core.eda.nokia.com/v1
kind: WorkflowDefinition
metadata:
  name: oam-ping-gvk
spec:
  image: registry.gitlabsr.nuq.ion.nokia.net/sr/eda/scripts/workflow-oam-ping:1.0.0
  imagePullSecrets:
  - gitlab-scripts
  flowDefinitionResource:
    group: oam.eda.nokia.com
    version: v1alpha1
    kind: Ping

Where:

image refers to the image to launch (as a Kubernetes Job).
imagePullSecrets contains any secrets to use to pull the image.
flowDefinitionResource contains the group, version, and kind to trigger the workflow against.

A WorkflowDefinition alone does not define inputs to the workflow however - rather indicating the kind to use as input. The CRD for this kind is loaded in using a Manifest as normal. To complete the example above, the Manifest would contain:


apiVersion: core.eda.nokia.com/v1
kind: Manifest
metadata:
  name: oam
spec:
-- snip --
  components:
  - cr:
      path: oam/workflows/ping/core_v1_flowdefinition.yaml
  - crd:
      api:
        expose: readWrite
      namespaced: true
      path: oam/crds/oam.eda.nokia.com_pings.yaml
      schema: oam/openapiv3/eda_oas_oam.eda.nokia.com_pings.json
      workflow: true   
-- snip --

Where:

The cr component contains the content above - the WorkflowDefinition.
The crd component contains a standard CRD, which is serialized as JSON and provided as input to the workflow on execution. The only addition is the workflow boolean - this indicates that this resource triggers workflows, and controls it appearing at the /workflows endpoint in the API, vs the /apps endpoint.

As a result, upon creation of the Ping kind, the registry.gitlabsr.nuq.ion.nokia.net/sr/eda/scripts/workflow-oam-ping:1.0.0 image is launched as a Job , with the spec of Ping passed as JSON input.

The workflow definitions page

The Workflow Definitions page in the EDA GUI allows you to view, delete, and sometimes edit and duplicate those workflow definitions that are known to EDA.

Note: For workflow definitions that are installed (and owned) by apps, only the View configuration and Delete actions are available. Duplicating and editing these workflow definitions is not permitted.

On this page you can also create new workflow definitions. Workflow definitions are a normal EDA resource and are defined by a Custom Resource Definition component of a Manifest, whose creation or modification you commit or add to a larger transaction like any other EDA resource.

Table 1. Elements of the Workflows page
#	Name	Function
1	Workflow definitions list	A list of workflow definitions known to EDA. Some of these workflow definitions may be created by users. Other, indicated by a lock icon, were added to EDA when specific applications were installed.
2	Actions menu	Displayed when clicking on the More icon at the right edge of the row for an individual workflow definition. Available actions are: Configuration view: opens the workflow definition view, displaying details of its metadata, specification, schema, and status. The view is read-only. Edit: like Configuration view, this opens the workflow definition view, but the configuration can be modified and any changes committed. This option is not available for workflow definitions owned by individual EDA applications. Duplicate: allows you to save a copy of the selected workflow definition with a new name. This option is not available for workflow definitions owned by individual EDA applications. Delete: Deletes the selected workflow definition, after confirmation.
	Information panel	When expanded, the information panel displays details about the selected workflow definition.
	Create button	Click to open the Definitions page to create and commit a new workflow definition.

Managing workflows with edactl

Some workflows may require user input to allow the workflow to proceed. You can use the following commands to handle workflows that require user input:

To display workflows awaiting input, use the following command:
```
edactl get waitforinputs 
```
To acknowledge a workflow and allow it to continue, use the following command:
```
edactl workflow ack <id>
```
For example, to acknowledge the workflow whose ID is 10:
```
edactl workflow ack 10
```
To terminate a workflow, use the following command
```
edactl workflow nack <id>
```
For example, to terminate the workflow whose ID is 20:
```
edactl workflow nack 20
```

You can also use edactl to query EDA regarding workflows.

To view all workflows:

edactl workflow get

For example:

edactl workflow get 
ID NAMESPACE NAME TYPE STATUS 
1 eda-system bulkapps-eda.nokia.com app-installer COMPLETED 
2 eda-system bulkapps-eda.nokia.com app-installer FAILED

To view details of a specific workflow:

edactl workflow get <id>

For example:

edactl workflow get 1 
ID: 1 
Namespace: eda-system 
Name: bulkapps-eda.nokia.com 
Status: COMPLETED 
Workflow Steps: 
↓ init 
↓ Fetching 
↓ Verifying 
↓ Committing 
↓ Applying 
↓ Installed

To view logs for a workflow:
```
edactl workflow logs <id> 
```
For Example: edactl workflow logs 20