Python
Python overview
Python is an established programming language that is widely used in systems administration. It has become one of the most prominent languages for network automation.
Python is popular because of its extensive libraries that software developers can import and enhance, and because it offers programming simplicity without diluting the flexibility and scalability offered by other languages.
Python in SR OS
SR OS uses version 2 and version 3 of the Python language.
Python 2 and Python 3 are available for ESM applications. Syslog supports Python 2 only. For more information, see Customizing syslog messages using Python and the 7450 ESS, 7750 SR, and VSR Triple Play Service Delivery Architecture Guide, "Python script support for ESM".
Python 3 is available for Python applications run in the MD-CLI using the pyexec or alias commands, and for applications executed remotely. Python 2 is not supported for pyexec, alias, CRON, or EHS in the MD‑CLI.
The remainder of this section references the SR OS Python 3 functionality only.
Prerequisites
The SR OS node must be in model-driven management interface configuration mode to run Python 3 applications.
To execute Python 3 applications on a remote device that connects to an SR OS device, the following prerequisites must be met:
On the SR OS node, the NETCONF protocol must be enabled.
The user must have the required permissions to use the NETCONF protocol.
On the remote device (that is, the device that executes the application), a Python 3 interpreter of version 3.10 or newer is required.
YANG model support
The Python 3 and pySROS features support Nokia YANG modules (combined and split) and OpenConfig YANG modules. If the YANG modules are to be obtained from the SR OS device, the YANG module files must be available at the specified schema path. See NETCONF monitoring for more information.
Python 3 interpreter
The Python interpreter is software that compiles and runs applications that developers create. It also handles importing additional libraries and all syntax for the language itself.
SR OS provides the MicroPython interpreter on the SR OS node. This interpreter implements Python 3.4 and is designed to operate using a small memory footprint, which allows SR OS to launch multiple interpreters in parallel.
The Python 3 interpreter is called by the SR OS function that requires it; for example, the MD-CLI. The interpreter is not called directly by the operator.
Building an application
Customized applications can be written to run on an SR OS node or remotely.
Developers can create applications using any development environment, including a Python 3-based Integrated Development Environment (IDE) or a text editor.
Python 3 applications written for SR OS are designed to be portable between systems (SR OS and operator devices), provided the libraries and language features used are available on both systems.
Available libraries
The following table describes the available preinstalled Python libraries that SR OS provides.
Library | Description |
---|---|
binutils |
Collection of binary utilities |
datetime |
Manipulation of various formats for date, time, and time zone. The strptime function is not supported. |
ubinascii |
Bidirectional translation between binary data and various ASCII encodings |
ujson |
Conversion between Python objects and JSON data format |
pysros |
Model-driven management API for SR OS (Python for the Service Router Operating System – pySROS) |
ucollections |
Advanced collection and container types to hold or accumulate various objects |
ure |
Simple regular expressions |
sys |
System-specific parameters and functions |
uhashlib |
Binary data hashing algorithms |
ustruct |
Pack and unpack primitive data types |
ipaddress |
IPv4 and IPv6 address manipulation |
uio |
Input and output streams |
utime |
Obtain current date and time, measuring intervals, and delays. This function has been updated to provide support for SR OS and is documented in the API documentation. |
The pySROS API
Python for the Service Router Operating System (pySROS) libraries provide the programmatic interface to the SR OS model-driven management framework.
The libraries provide a set of methods for developers to obtain, manipulate, and deliver data to and from an SR OS node.
Installing the pySROS libraries
The pySROS libraries are delivered and preinstalled on SR OS nodes. This libraries can be installed on another device, such as a PC or UNIX workstation.
Use one of the following methods to install the pySROS libraries:
from the Python Package Index (PyPI) using the pip tool (preferred method)
To do this, execute the pip install pysros command on the device.
from source code delivered from github.com
To do this, clone the source code repository to your device by executing the git clone https://github.com/nokia/pysros command.
The source code can then be compiled and installed by executing python3 setup.py install.
from source code delivered from the Nokia online support portal
To do this, obtain the source code from the Nokia online support portal. This is the same location where the SR OS software resides.
The source code can then be compiled and installed by executing python3 setup.py install.
Importing into applications
The pySROS libraries must be imported to use the provided API in developed applications.
The libraries may be imported using the import pysros command, or only specific elements of the API may be imported; for example, from pysros.management import connect.
API documentation
This chapter provides a high-level overview about how to connect to the model-driven interfaces of an SR OS device and manipulate simple data structures. Detailed technical documentation for the pySROS API is provided within the source code bundle in the docs directory.
The API documentation is provided as source code and should be built as needed. To compile the API documentation, run the following commands from the docs directory:
pip install -r requirements.txt
make html
The API documentation is compiled into the build/html folder in HTML format.
Connecting to a model-driven SR OS device
To enable access to an SR OS device, the user must create a Connection object, which is a Python object that references the application’s connection to a specific SR OS node.
The underlying connection transport protocol is NETCONF.
The connect() API method is provided to create this Connection object.
The following example shows the creation of a connection named connection_object to a device with the IP address 192.168.168.1.
from pysros.management import connect
connection_object = connect(host='192.168.168.1', username='admin', password='admin')
The following table describes the arguments available for the connect() API method.
Argument | Type | Mandatory | Description |
---|---|---|---|
host |
string |
Yes |
IP address, hostname, or FQDN of the SR OS device to connect to |
username |
string |
Yes |
Username to connect to the SR OS device |
password |
string |
Yes |
Password to connect to the SR OS device If no password is provided, SSH key-based authentication is attempted. |
port |
integer |
No |
TCP port on the SR OS node that the connect attempt is made to Default: 830 |
yang_directory |
string |
No |
Directory to read YANG module files from Default: None |
rebuild |
Boolean |
No |
Forces the schema cache to be rebuilt upon connect Default: False |
timeout |
integer |
No |
Maximum time for connection, obtaining YANG modules, and schema generation Default: 300 seconds |
hostkey_verify |
Boolean |
No |
Disables SSH hostkey checking. Nokia recommends to not use this in a live deployment. |
When a Python application makes a connection to an SR OS device using the connect() method, the application performs the following actions.
The device is queried to identify a set of modules required to manage the device. This is performed by identifying the module-set-id (yang-library:1.0) or content-id (yang-library:1.1) from the devices advertised capabilities.
The set of modules are queried using /modules-state from the IETF YANG library RFC 8525.
The local schema cache is checked to determine whether a model-driven management schema is already generated for the same set of YANG modules. If no identical local schema cache exists, or if the rebuild argument is set to True, the following occurs.
If the yang_directory argument is not provided to the connect() API method, each YANG module listed by the SR OS node is downloaded directly from the node.
If the yang_directory argument is provided to the connect() API method, the YANG module files from this directory that correspond to the list of required modules obtained from the router are used.
The YANG modules are compiled into a schema cache for use by the developer and the pySROS libraries.
The local model-driven schema cache is stored in the .pysros directory within the developer’s local home directory. If this directory is deleted or modified, the cache is rebuilt on the next connect() API call.
Obtaining modeled data
Use the get() API method to obtain modeled data from an SR OS device.
The get() API method accepts as inputs a JSON instance identifier path and a datastore. The method returns a Python data structure, which uses the pySROS format to provide modeled data to the developer. See The pySROS data structure for more information.
A JSON instance path is obtained from an SR OS node using the pwc json-instance-path command. See the 7450 ESS, 7750 SR, 7950 XRS, and VSR MD-CLI User Guide for more information.
The following example obtains data from an SR OS node Connection object.
my_data = connection_object.running.get('/nokia-state:state/system/oper-name')
Performing operations with pySROS
In addition to device configuration and state information collection, operators can perform the following types of node operations:
- a YANG-modeled operation (defined as an 'action' in a YANG model), which uses structured data for both input and output
- an MD-CLI command, which provides semi-structured input and unstructured output
Nokia recommends using the action() API method when performing operations with SR OS. Some cases may require using an unmodeled MD-CLI command, which can be performed using the cli() API method and manipulating the resulting string output.
YANG-modeled operations with pySROS
The action() API method allows developers to pass YANG-modeled, structured-data inputs to an operation to be executed on the router and to receive YANG-modeled, structured-data output.
The following example shows a ping operation from the pySROS API.
>>> connection_object.action('/nokia-oper-global:global-operations/ping', {'destination': '192.168.122.1'})
Action({'operation-id': Leaf(13), 'start-time': Leaf('2023-10-04T19:00:20.1Z'), 'results': Container({'test-parameters': Container({'destination': Leaf('192.168.122.1'), 'bypass-routing': Leaf(False), 'router-instance': Leaf('Base'), 'srv6-policy': Leaf(False), 'candidate-path': Leaf(False), 'preference': Leaf(100), 'count': Leaf(5), 'output-format': Leaf('detail'), 'do-not-fragment': Leaf(False), 'fc': Leaf('nc'), 'interval': Leaf('1'), 'pattern': Leaf('sequential'), 'size': Leaf(56), 'timeout': Leaf(5), 'tos': Leaf(0), 'ttl': Leaf(64)}), 'probe': {1: Container({'probe-index': Leaf(1), 'status': Leaf('no-route-to-dest')}), 2: Container({'probe-index': Leaf(2), 'status': Leaf('no-route-to-dest')}), 3: Container({'probe-index': Leaf(3), 'status': Leaf('no-route-to-dest')}), 4: Container({'probe-index': Leaf(4), 'status': Leaf('no-route-to-dest')}), 5: Container({'probe-index': Leaf(5), 'status': Leaf('no-route-to-dest')})}, 'summary': Container({'statistics': Container({'packets': Container({'sent': Leaf(5), 'received': Leaf(0), 'loss': Leaf('100.0')}), 'round-trip-time': Container({'minimum': Leaf(0), 'average': Leaf(0), 'maximum': Leaf(0)})})})}), 'status': Leaf('completed'), 'end-time': Leaf('2023-10-04T19:00:24.3Z')})
MD-CLI command execution with pySROS
The cli() API method allows developers to pass a single line string containing a single MD-CLI command (including output modifiers, if required) to an existing SR OS connection object. SR OS executes this command immediately and returns the output as a string. See NETCONF operations using the md-cli-raw-command request for more information about supported commands.
The following example executes an MD-CLI command from the pySROS API.
output_string = connection_object.cli('show system time')
print(output_string)
===============================================================================
Date & Time
===============================================================================
Current Date & Time : 2021/09/15 20:27:27 DST Active : no
Current Zone : UTC Offset from UTC : 00:00
-------------------------------------------------------------------------------
Non-DST Zone : UTC Offset from UTC : 00:00
Zone type : standard
-------------------------------------------------------------------------------
No DST zone configured.
-------------------------------------------------------------------------------
Prefer Local Time : NO
===============================================================================
Editing configuration
Python data structures, in pySROS format, and specific values within the data structures can be configured on an SR OS device.
The set() API method is provided to configure a node. The set() API method aggregates several model-driven transactional operations into a single method:
configuration of the node (performed in private candidate configuration mode)
update, validate, and commit
The set() API method accepts as input a JSON instance path, a datastore, and a payload. The payload may be a single value, a key and value pair, a crafted Python data structure following the pySROS data-structure format, or a pySROS data structure object received from a get() API method call.
The following examples show how to set the system name to ‟hostname”.
connection_object.candidate.set("/nokia-conf:configure/system/name", "hostname")
connection_object.candidate.set("/nokia-conf:configure/system", {‛name’:‛hostname’})
connection_object.candidate.set("/nokia-
conf:configure", {‛system’: {‛name’: ‛hostname’}})
The pySROS data structure
The key to efficient Python programs for network management is the ability to manipulate data in a consumable format.
The pySROS libraries ensure that the YANG modeled data from SR OS is supplied in native Python data structures that are simple to understand and handle in code.
When data is obtained from an SR OS node, it is returned in a specific data structure format, a pySROS data structure.
The following table shows the pySROS conversion from model-driven YANG-based data structures to native Python data structures.
YANG structure | Python 3 structure |
---|---|
Container |
Dict |
Leaf |
Value (Type derived as shown in The pySROS data structure) |
Leaf-list |
List |
List |
Dict |
User-ordered List |
OrderedDict keyed on the YANG list key value |
The pySROS libraries provide Python class wrappers around each node type to assist with data manipulation. This information can be used with some of the features built into the pySROS API, such as pretty-printing.
Containers are wrapped in a Container() class.
Leafs are wrapped in a Leaf() class.
Command usage to obtain the value of a leaf
The following example shows how to obtain the value of the leaf that is wrapped in a Leaf() class by calling the .data function on the leaf.
from pysros.wrappers import Leaf
obj = Leaf('example')
print(obj.data)
Leaf-lists are wrapped in a LeafList() class.
All pySROS class wrappers for YANG nodes are provided by pysros.wrappers.
YANG schema metadata can be obtained on a specific data object by calling the schema function. The YANG metadata that is available within the pySROS data structure (data is the specific pySROS formatted data object) is the data.schema.module. This is the YANG module name the element comes from. If this element in the YANG schema is imported or included from another YANG modules, the root YANG module is displayed. If the YANG modules where the element is found is an augment to another module, the augment module is shown.
YANG types are also converted into native Python types. The following table describes the rules for this conversion.
Base YANG type | Python 3 type |
---|---|
binary |
String |
bits |
String |
boolean |
Boolean |
decimal64 |
String |
empty |
|
enumeration |
String |
identityref |
String |
int8 |
Integer |
int16 |
Integer |
int32 |
Integer |
int64 |
Integer |
leafref |
(not applicable2) |
string |
String |
uint8 |
Integer |
uint16 |
Integer |
uint32 |
Integer |
uint64 |
Integer |
union |
String3 |
Provisioning and precompiling Python applications
Python applications can be configured and executed in the MD-CLI. This allows administrators to provide specific Python applications to users.
Python applications can be referenced in configuration statements, such as MD-CLI aliases.
Python applications are configured in the configure python context. The following is a Python application configuration example.
(ex)[/configure python]
A:admin@node-2# info
python-script "my_example" {
admin-state enable
urls ["cf3:\example.py" "ftp://user:password@192.168.168.254/example.py"]
version python3
}
After a Python application is configured and committed, the SR OS node attempts to load the files specified in the urls field in the order they are listed. If the first URL location can be reached and a file containing valid Python code is found, the system stores this source code ready for use.
If the first URL cannot be reached or the file does not contain valid source code, the system moves to the next URL in the list. If no URL locations contain valid Python source code, the oper-state leaf for the configured application is set to down in the following context.
state python python-script application_name
Refreshing configured python scripts
When defined in the SR OS configuration, Python scripts are stored in memory on the SR OS node. The URLs defined in the configuration are not rechecked, unless it is specifically requested by an operator.
If a new version of the script is placed in the URL reference location, whether locally on the SR OS nodes storage card or remotely, the tools perform python-script reload application_name command must be run to use the new version.
Execution pathways
This section describes the methods that are provided to execute Python applications.
Executing a Python application remotely
The pySROS libraries provide the ability to interface with one or more SR OS devices from a device that has a Python interpreter installed. This execution pathway is not specific to SR OS (beyond the use of the pySROS libraries) and is not described further in this guide.
Executing a Python application from the command line
The MD-CLI provides the ability to execute a Python application directly from the command line.
The pyexec command takes as an option, either the name of a Python application from the SR OS configuration (configure python python-script application_name) or the URL to the location (local/remote) of a Python application.
For example, if the pyexec application_name command is executed, the SR OS performs the following steps.
The SR OS device searches for a configuration element configure python python-script application_name. If a configuration element is found, the system attempts to compile and execute this script. A failure ends the execution.
The SR OS device treats the application_name as a URL. As application_name does not have a URL identifier (such as cf3:\ or ftp://) attached to it, SR OS prepends the present working directory of the system (which is cf3:\ by default on most systems).
SR OS attempts to read from the derived URL; for example, cf3:\application_name. If this URL is successfully located, SR OS compiles and runs the application. A failure ends the execution.
Each time pyexec is invoked, a new Python interpreter is spawned.
pyexec and input parameters
The pyexec command can accept a maximum of ten input arguments provided using the MD-CLI command line. The Python application developer must handle these as standard input (STDIN) arguments.
Executing a Python application as an output modifier
Python applications can be entered as an output modifier to the MD-CLI commands.
The following is an example output from an info command that is processed by a Python application, which capitalizes the output.
(ex)[/]
A:admin@node-2# file show cf3:\capitals.py
File: capitals.py
-------------------------------------------------------------------------------
import sys
def main():
for line in sys.stdin:
print(line.upper(), end="")
main()
===============================================================================
(ex)[/]
A:admin@node-2# show version | pyexec capitals.py
TIMOS-B-21.7.B1-9 BOTH/X86_64 NOKIA 7750 SR COPYRIGHT (C) 2000-2021 NOKIA.
ALL RIGHTS RESERVED. ALL USE SUBJECT TO APPLICABLE LICENSE AGREEMENTS.
BUILT ON THU JUN 12 18:35:55 PDT 2021 BY BUILDER IN /BUILDS/C/217B/B1-9/PANOS/MAIN/SROS
Chaining Python applications
The pyexec command supports chaining multiple applications on the command line. Python applications called with the pyexec command can produce data on the standard output (STDOUT) stream and consume data on the standard input (STDIN) stream. Multiple Python applications can then be chained together.
Python applications and the MD-CLI pager
Python applications executed using the pyexec command use the built-in MD-CLI pager to display the output one page at a time. This behavior can be disabled per Python application using the no-more output modifier, or globally by setting the MD-CLI environment more option to false, as required.
When the MD-CLI display output is paused by the pager while executing a Python application, pressing q closes the pager and resumes executing the Python application. Pressing Ctrl-c closes the pager and terminates the Python application.
Executing Python applications with MD-CLI command aliases
Python applications can be executed from a configured alias in MD-CLI. See ‟Command Aliases” in the 7450 ESS, 7750 SR, 7950 XRS, and VSR MD-CLI User Guide. for more information about how to configure an alias in the MD-CLI.
Aliases may call a configured (named) Python application or the pyexec command.
There are specific AAA considerations for each method. See Authentication, authorization, and accounting for more information.
Each time an alias that references a Python application is executed, a new Python interpreter is spawned.
Security
Python authentication and authorization
Python applications are dependent on authentication, authorization, and accounting in the same way as any other feature of SR OS. See Authentication, authorization, and accounting for more information.
The user associated with the current session is used for authentication, authorization, and accounting of the script, as well as the authorization of the content of the scripts results, and any resulting file access.
If the configure python python-script run-as-user command is configured, the specified user is used for all authentication, authorization, accounting, and file access of the script, instead of the current session user.
The run-as-user command is configured on a per python-script basis and takes precedence over any other user configured on a less granular level (for example, CRON or EHS).
Authentication
To make a connection to the model-driven interface of an SR OS device, user authentication is required.
User authentication is deemed to have already occurred if the Python application is executed directly on the SR OS node.
If the Python application is executed remotely, user credentials must be provided to the connect() API call.
Authorization
To successfully use a Python application, the authenticated user must have authorization to both execute the application and perform each of the model-driven operations contained within the Python application.
Other restrictions
To protect the integrity of an SR OS system, a number of facilities are disabled in Python while executing an application on SR OS. These include, but are not limited to the following:
importing additional libraries outside of the supported list
creating, reading, writing, or otherwise manipulating network sockets
garbage collection
debugging libraries
Memory management
SR OS does not allow a Python application to start if there is insufficient memory available at the point of execution, or if too many Python applications are running concurrently.
When a Python 3 application is spawned, SR OS confirms that the minimum percentage of sufficient free system memory is available. The percentage value for minimum available system memory is configurable within the MD-CLI environment settings.
The Python 3 application then reserves memory incrementally (and automatically) for the application up to the value of the maximum memory configured in the MD-CLI environment settings.
Performance and scale
The ability to execute Python applications conveys no expectation of a specific performance or scale level.