Python

Python overview

Python is an established programming language that is widely used in systems administration. It has become one of the most prominent languages for network automation.

Python is popular because of its extensive libraries that software developers can import and enhance, and because it offers programming simplicity without diluting the flexibility and scalability offered by other languages.

Note: This chapter assumes prior knowledge and experience with the Python programming language.

Python in SR OS

SR OS uses version 2 and version 3 of the Python language.

Python 2 and Python 3 are available for ESM applications. Syslog supports Python 2 only. For more information, see Customizing syslog messages using Python and the 7450 ESS, 7750 SR, and VSR Triple Play Service Delivery Architecture Guide, "Python script support for ESM".

Python 3 is available for Python applications run in the MD-CLI using the pyexec or alias commands, and for applications executed remotely. Python 2 is not supported for pyexec, alias, CRON, or EHS in the MD‑CLI.

The remainder of this section references the SR OS Python 3 functionality only.

Prerequisites

The SR OS node must be in model-driven management interface configuration mode to run Python 3 applications.

To execute Python 3 applications on a remote device that connects to an SR OS device, the following prerequisites must be met.

  • On the SR OS node, the NETCONF protocol must be enabled.

  • The user must have the required permissions to use the NETCONF protocol.

  • On the remote device (that is, the device that executes the application), a Python 3 interpreter of version 3.6 or newer is required.

YANG model support

The Python 3 and pySROS features support Nokia YANG modules (combined and split) and OpenConfig YANG modules. If the YANG modules are to be obtained from the SR OS device, the YANG module files must be available at the specified schema path. See NETCONF monitoring for more information.

Python 3 interpreter

The Python interpreter is software that compiles and runs applications that developers create. It also handles importing additional libraries and all syntax for the language itself.

SR OS provides the MicroPython interpreter on the SR OS node. This interpreter implements Python 3.4 and is designed to operate using a small memory footprint, which allows SR OS to launch multiple interpreters in parallel.

The Python 3 interpreter is called by the SR OS function that requires it; for example, the MD-CLI. The interpreter is not called directly by the operator.

Building an application

Customized applications can be written to run on an SR OS node or remotely.

Developers can create applications using any development environment, including a Python 3-based Integrated Development Environment (IDE) or a text editor.

Python 3 applications written for SR OS are designed to be portable between systems (SR OS and operator devices), provided the libraries and language features used are available on both systems.

Available libraries

The following table describes the available preinstalled Python libraries that SR OS provides.

Note: Libraries prefixed by the letter ‟u” are MicroPython-specific libraries. Importing to a script using the base name is typically possible; for example, import json imports ujson when running on an SR OS device.
Table 1. Available libraries
Library Description

binutils

Collection of binary utilities

datetime

Manipulation of various formats for date, time, and time zone. The strptime function is not supported.

ubinascii

Bidirectional translation between binary data and various ASCII encodings

ujson

Conversion between Python objects and JSON data format

pysros

Model-driven management API for SR OS (Python for the Service Router Operating System – pySROS)

ucollections

Advanced collection and container types to hold or accumulate various objects

ure

Simple regular expressions

sys

System-specific parameters and functions

uhashlib

Binary data hashing algorithms

ustruct

Pack and unpack primitive data types

ipaddress

IPv4 and IPv6 address manipulation

uio

Input and output streams

utime

Obtain current date and time, measuring intervals, and delays. This function has been updated to provide support for SR OS and is documented in the API documentation.

The pySROS API

Python for the Service Router Operating System (pySROS) libraries provide the programmatic interface to the SR OS model-driven management framework.

The libraries provide a set of methods for developers to obtain, manipulate, and deliver data to and from an SR OS node.

Note: Nokia recommends reviewing the API documentation provided with the pySROS libraries before developing any Python applications.

Installing the pySROS libraries

The pySROS libraries are delivered and preinstalled on SR OS nodes. This libraries can be installed on another device, such as a PC or UNIX workstation.

Note: A Python 3 interpreter must be already installed on the device.

Use one of the following methods to install the pySROS libraries:

  1. from the Python Package Index (PyPI) using the pip tool (preferred method)

    To do this, execute the pip install pysros command on the device.

  2. from source code delivered from github.com

    To do this, clone the source code repository to your device by executing the git clone https://github.com/nokia/pysros command.

    The source code can then be compiled and installed by executing python3 setup.py install.

  3. from source code delivered from the Nokia online support portal

    To do this, obtain the source code from the Nokia online support portal. This is the same location where the SR OS software resides.

    The source code can then be compiled and installed by executing python3 setup.py install.

Importing into applications

The pySROS libraries must be imported to use the provided API in developed applications.

The libraries may be imported using the import pysros command, or only specific elements of the API may be imported; for example, from pysros.management import connect.

API documentation

This chapter provides a high-level overview about how to connect to the model-driven interfaces of an SR OS device and manipulate simple data structures. Detailed technical documentation for the pySROS API is provided within the source code bundle in the docs directory.

The API documentation is provided as source code and should be built as needed. To compile the API documentation, run the following commands from the docs directory:

  • pip install -r requirements.txt

  • make html

The API documentation is compiled into the build/html folder in HTML format.

Note: Nokia recommends reviewing the API documentation provided with the pySROS libraries before developing any Python applications.

Connecting to a model-driven SR OS device

To enable access to an SR OS device, the user must create a Connection object, which is a Python object that references the application’s connection to a specific SR OS node.

The underlying connection transport protocol is NETCONF.

The connect() method is provided to create this Connection object.

The following example shows the creation of a connection named connection_object to a device with the IP address 192.168.168.1.

from pysros.management import connect
connection_object = connect(host='192.168.168.1', username='admin', password='admin')

The following table describes the arguments available for the connect() method.

Note: All arguments are ignored when the Python application is executed on an SR OS device.
Table 2. Connect() method arguments
Argument Type Mandatory Description

host

string

Yes

IP address, hostname, or FQDN of the SR OS device to connect to

username

string

Yes

Username to connect to the SR OS device

password

string

Yes

Password to connect to the SR OS device

If no password is provided, SSH key-based authentication is attempted.

port

integer

No

TCP port on the SR OS node that the connect attempt is made to

Default: 830

yang_directory

string

No

Directory to read YANG module files from

Default: None

rebuild

Boolean

No

Forces the schema cache to be rebuilt upon connect

Default: False

timeout

integer

No

Maximum time for connection, obtaining YANG modules, and schema generation

Default: 300 seconds

hostkey_verify

Boolean

No

Disables SSH hostkey checking. Nokia recommends to not use this in a live deployment.

When a Python application makes a connection to an SR OS device using the connect() method, the application performs the following actions.

  • The device is queried to identify a set of modules required to manage the device. This is performed by identifying the module-set-id (yang-library:1.0) or content-id (yang-library:1.1) from the devices advertised capabilities.

  • The set of modules are queried using /modules-state from the IETF YANG library RFC 8525.

  • The local schema cache is checked to determine whether a model-driven management schema is already generated for the same set of YANG modules. If no identical local schema cache exists, or if the rebuild argument is set to True, the following occurs.

    • If the yang_directory argument is not provided to the connect() method, each YANG module listed by the SR OS node is downloaded directly from the node.

    • If the yang_directory argument is provided to the connect() method, the YANG module files from this directory that correspond to the list of required modules obtained from the router are used.

  • The YANG modules are compiled into a schema cache for use by the developer and the pySROS libraries.

Note: The connect method is slower to complete for the first compilation of a specific set of YANG modules. Subsequent connections for that set of YANG modules are significantly faster. A locally cached set of YANG modules is not specific to an individual router.

The local model-driven schema cache is stored in the .pysros directory within the developer’s local home directory. If this directory is deleted or modified, the cache is rebuilt on the next connect() API call.

Obtaining modeled data

Use the get() method to obtain modeled data from an SR OS device.

The get() method accepts as inputs a JSON instance identifier path and a datastore. The method returns a Python data structure, which uses the pySROS format to provide modeled data to the developer. See The pySROS data structure for more information.

A JSON instance path is obtained from an SR OS node using the pwc json-instance-path command. See the 7450 ESS, 7750 SR, 7950 XRS, and VSR MD-CLI User Guide for more information.

The following example obtains data from an SR OS node Connection object.

my_data = connection_object.running.get('/nokia-state:state/system/oper-name')
Note: State information is obtained from the running the datastore when using the pySROS libraries.

Executing MD-CLI commands

Nokia recommends using the pySROS API when interacting with SR OS using model-driven interfaces. However, some cases may require an unmodeled MD-CLI command, which the developer can get by manipulating the resulting string output.

The cli() API method allows developers to pass a single line string containing a single MD-CLI command (including output modifiers, if required) to an existing SR OS connection object. This command is executed immediately and the output is returned as a string. See NETCONF operations using the md-cli-raw-command request for more information about supported commands.

The following example executes an MD-CLI command from the pySROS API.

output_string = connection_object.cli('show system time')
print(output_string)
===============================================================================
Date & Time
===============================================================================
Current Date & Time : 2021/09/15 20:27:27    DST Active            : no
Current Zone        : UTC                    Offset from UTC       : 00:00
-------------------------------------------------------------------------------
Non-DST Zone        : UTC                    Offset from UTC       : 00:00
Zone type           : standard
-------------------------------------------------------------------------------
No DST zone configured.
-------------------------------------------------------------------------------
Prefer Local Time   : NO
===============================================================================

Editing configuration

Python data structures, in pySROS format, and specific values within the data structures can be configured on an SR OS device.

The set() method is provided to configure a node. The set() method aggregates several model-driven transactional operations into a single method:

  • configuration of the node (performed in private candidate configuration mode)

  • update, validate, and commit

The set() method accepts as input a JSON instance path, a datastore, and a payload. The payload may be a single value, a key and value pair, a crafted Python data structure following the pySROS data-structure format, or a pySROS data structure object received from a get() method call.

The following examples show how to set the system name to ‟hostname”.

connection_object.candidate.set("/nokia-conf:configure/system/name", "hostname")
connection_object.candidate.set("/nokia-conf:configure/system", {‛name’:‛hostname’})
connection_object.candidate.set("/nokia-
conf:configure", {‛system’: {‛name’: ‛hostname’}})

The pySROS data structure

The key to efficient Python programs for network management is the ability to manipulate data in a consumable format.

The pySROS libraries ensure that the YANG modeled data from SR OS is supplied in native Python data structures that are simple to understand and handle in code.

When data is obtained from an SR OS node, it is returned in a specific data structure format, a pySROS data structure.

The following table shows the pySROS conversion from model-driven YANG-based data structures to native Python data structures.

Table 3. Model-driven to Python data structure conversion rules
YANG structure Python 3 structure

Container

Dict

Leaf

Value (Type derived as shown in The pySROS data structure)

Leaf-list

List

List

Dict

User-ordered List

OrderedDict keyed on the YANG list key value

The pySROS libraries provide Python class wrappers around each node type to assist with data manipulation. This information can be used with some of the features built into the pySROS API, such as pretty-printing.

Containers are wrapped in a Container() class.

Leafs are wrapped in a Leaf() class.

Command usage to obtain the value of a leaf

The following example shows how to obtain the value of the leaf that is wrapped in a Leaf() class by calling the .data function on the leaf.

from pysros.wrappers import Leaf
obj = Leaf('example')
print(obj.data)

Leaf-lists are wrapped in a LeafList() class.

All pySROS class wrappers for YANG nodes are provided by pysros.wrappers.

YANG schema metadata can be obtained on a specific data object by calling the schema function. The YANG metadata that is available within the pySROS data structure (data is the specific pySROS formatted data object) is the data.schema.module. This is the YANG module name the element comes from. If this element in the YANG schema is imported or included from another YANG modules, the root YANG module is displayed. If the YANG modules where the element is found is an augment to another module, the augment module is shown.

YANG types are also converted into native Python types. The following table describes the rules for this conversion.

Table 4. YANG types
Base YANG type Python 3 type

binary

String

bits

String

boolean

Boolean

decimal64

String

empty

enumeration

String

identityref

String

int8

Integer

int16

Integer

int32

Integer

int64

Integer

leafref

(not applicable2)

string

String

uint8

Integer

uint16

Integer

uint32

Integer

uint64

Integer

union

String3

Provisioning and precompiling Python applications

Python applications can be configured and executed in the MD-CLI. This allows administrators to provide specific Python applications to users.

Python applications can be referenced in configuration statements, such as MD-CLI aliases.

Python applications are configured in the configure python context. The following is a Python application configuration example.

(ex)[/configure python]
A:admin@sros# info
    python-script "my_example" {
        admin-state enable
        urls ["cf3:\example.py" "ftp://user:password@192.168.168.254/example.py"]
        version python3
    }

After a Python application is configured and committed, the SR OS node attempts to load the files specified in the urls field in the order they are listed. If the first URL location can be reached and a file containing valid Python code is found, the system stores this source code ready for use.

If the first URL cannot be reached or the file does not contain valid source code, the system moves to the next URL in the list. If no URL locations contain valid Python source code, the oper-state leaf for the configured application is set to down in the following context.

state python python-script application_name

Refreshing configured python scripts

When defined in the SR OS configuration, Python scripts are stored in memory on the SR OS node. The URLs defined in the configuration are not rechecked, unless it is specifically requested by an operator.

If a new version of the script is placed in the URL reference location, whether locally on the SR OS nodes storage card or remotely, the tools perform python-script reload application_name command must be run to use the new version.

Execution pathways

This section describes the methods that are provided to execute Python applications.

Running a Python application remotely

The pySROS libraries provide the ability to interface with one or more SR OS devices from a device that has a Python interpreter installed. This execution pathway is not specific to SR OS (beyond the use of the pySROS libraries) and is not described further in this guide.

Running a Python application from the command line

The MD-CLI provides the ability to execute a Python application directly from the command line.

The pyexec command takes as an option, either the name of a Python application from the SR OS configuration (configure python python-script application_name) or the URL to the location (local/remote) of a Python application.

For example, if the pyexec application_name command is executed, the SR OS performs the following steps.

  1. The SR OS device searches for a configuration element configure python python-script application_name. If a configuration element is found, the system attempts to compile and execute this script. A failure ends the execution.

  2. The SR OS device treats the application_name as a URL. As application_name does not have a URL identifier (such as cf3:\ or ftp://) attached to it, SR OS prepends the present working directory of the system (which is cf3:\ by default on most systems).

  3. SR OS attempts to read from the derived URL; for example, cf3:\application_name. If this URL is successfully located, SR OS compiles and runs the application. A failure ends the execution.

    Each time pyexec is invoked, a new Python interpreter is spawned.

pyexec and input parameters

The pyexec command can accept a maximum of ten input arguments provided using the MD-CLI command line. The Python application developer must handle these as standard input (STDIN) arguments.

Running a Python application as an output modifier

Python applications can be entered as an output modifier to the MD-CLI commands.

The following is an example output from an info command that is processed by a Python application, which capitalizes the output.

(ex)[/]
A:admin@node-2# file show cf3:\capitals.py
File: capitals.py
-------------------------------------------------------------------------------
import sys
def main():
  for line in sys.stdin:
    print(line.upper(), end="")
main()
===============================================================================
(ex)[/]
A:admin@node-2# show version | pyexec capitals.py
TIMOS-B-21.7.B1-9 BOTH/X86_64 NOKIA 7750 SR COPYRIGHT (C) 2000-2021 NOKIA.
ALL RIGHTS RESERVED. ALL USE SUBJECT TO APPLICABLE LICENSE AGREEMENTS.
BUILT ON THU JUN 12 18:35:55 PDT 2021 BY BUILDER IN /BUILDS/C/217B/B1-9/PANOS/MAIN/SROS

Chaining Python applications

The pyexec command supports chaining multiple applications on the command line. Python applications called with the pyexec command can produce data on the standard output (STDOUT) stream and consume data on the standard input (STDIN) stream. Multiple Python applications can then be chained together.

Note: A maximum of three pyexec commands may be chained together.

Python applications and the MD-CLI pager

Python applications executed using the pyexec command use the built-in MD-CLI pager to display the output one page at a time. This behavior can be disabled per Python application using the no-more output modifier, or globally by setting the MD-CLI environment more option to false, as required.

When the MD-CLI display output is paused by the pager while executing a Python application, pressing q closes the pager and resumes executing the Python application. Pressing Ctrl-c closes the pager and terminates the Python application.

Running Python applications using aliased MD-CLI

Python applications can be executed from a configured alias in MD-CLI. See ‟Command Aliases” in the 7450 ESS, 7750 SR, 7950 XRS, and VSR MD-CLI User Guide. for more information about how to configure an alias in the MD-CLI.

Aliases may call a configured (named) Python application or the pyexec command.

There are specific AAA considerations for each method. See Authentication, authorization, and accounting for more information.

Each time an alias that references a Python application is executed, a new Python interpreter is spawned.

Security

SR OS ensures that the routing, forwarding, and management functions of a device are protected and prioritized.

Python authentication and authorization

Python applications are dependent on authentication, authorization, and accounting in the same way as any other feature of SR OS. See Authentication, authorization, and accounting for more information.

The user associated with the current session is used for authentication, authorization, and accounting of the script, as well as the authorization of the content of the scripts results, and any resulting file access.

If the configure python python-script run-as-user command is configured, the specified user is used for all authentication, authorization, accounting, and file access of the script, instead of the current session user.

The run-as-user command is configured on a per python-script basis and takes precedence over any other user configured on a less granular level (for example, CRON or EHS).

Note: If the run-as-user command is specified, Nokia recommends paying particular attention to the authorization rules about who is allowed to execute the configured python-script.

Authentication

To make a connection to the model-driven interface of an SR OS device, user authentication is required.

User authentication is deemed to have already occurred if the Python application is executed directly on the SR OS node.

If the Python application is executed remotely, user credentials must be provided to the connect() API call.

Authorization

To successfully use a Python application, the authenticated user must have authorization to both execute the application and perform each of the model-driven operations contained within the Python application.

Other restrictions

To protect the integrity of an SR OS system, a number of facilities are disabled in Python while executing an application on SR OS. These include, but are not limited to the following:

  • importing additional libraries outside of the supported list

  • creating, reading, writing, or otherwise manipulating network sockets

  • garbage collection

  • debugging libraries

Memory management

SR OS does not allow a Python application to start if there is insufficient memory available at the point of execution, or if too many Python applications are running concurrently.

When a Python 3 application is spawned, SR OS confirms that the minimum percentage of sufficient free system memory is available. The percentage value for minimum available system memory is configurable within the MD-CLI environment settings.

The Python 3 application then reserves memory incrementally (and automatically) for the application up to the value of the maximum memory configured in the MD-CLI environment settings.

Performance and scale

The ability to execute Python applications conveys no expectation of a specific performance or scale level.

Note: Nokia recommends using smaller data sets, where possible, when executing a Python application on an SR OS device. For larger data sets, Nokia recommends executing Python applications remotely.
1

The specific type is provided by the pySROS libraries.

2

A leafref takes the YANG native type of the leaf it is referencing. This type is converted to Python according to the table.

3

A union YANG type may be a union of different YANG types, for example, a union of a string and a Boolean. As it is not possible to identify the intention at the time of obtaining the data, automatic type selection is not performed. Every union is treated as a string, allowing the developer to cast the element into a specified type.